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PREFACE 


Throughout  the  text  of  this  paper,  reference  is  made  to  volumes  1  through  V.  These  volumes  have  been 
published  as  separate  technical  papers  identified  as  follows: 

Volume  1 

Baum.  1  >.!{..  Modrick.  J.A..  &  Hollingsworth.  S.R.  Team  training  for  command  and  control  systems: 
Status.  AFIIIIL-TP-JIA".  Viright-Palterson  AFII.  OH:  Logistics  and  Technical  Training  Division.  Air 
Force  Human  Resources  Laboratorv.  April  1982. 

Volume  II 

Modrick.  J.  A..  Baum.  D.R..  v\  Hollingsworth.  S.R.  Tram  training  for  command  and  control  systems: 
Recommendations  for  research  program.  AFHRL-TP-82-8.  Vi  right-Palterson  AFB.  OH:  Logistics  and 
Technical  Training  Division.  Air  Force  Human  Resources  Laboratorv.  April  1982. 

Volume  III 

Baum.  D.JL.  Modrick.  J.A..  Hollingsworth.  S.R.  learn  training  for  command  and  control  systems- 
Recommendations  for  application  of  current  technology.  AF’HRL-TP-82-9.  Wright-Patterson  AFB. 
OH:  Logistics  and  Technical  Training  Division.  Air  Force  Human  Resources  Laboratorv.  April  1982. 

Volume  IV 

Hollingsworth.  S.R..  Modrick.  J.A..  &  Baum.  D.R.  Team  training  for  command  and  control  systems: 
Recommendations  for  simulation  facility.  AFHRL-TP-82- 10.  Vl  right-Patterson  AFB.  OIL  Logistics 
and  Technical  Training  Division.  Air  F'orce  Human  Resources  Laboratory.  April  1982. 

Volume  V 

Baum.  D.R..  Modrick.  J.A..  &  Hollingsworth.  S.R.  Team  training  for  command  and  control  systems: 
Executive  summary.  AFHRL-TP-82- 1 1.  Wright-Pallerson  AF'B.  OH:  Logistics  and  Technical  Training 
Division.  Air  Force  Human  Resources  Laboratory.  April  1982. 

This  paper  is  the  fifth  of  five  volumes  prepared  by  Honeywell  to  document  the  results  of  a  research 
program  to  evaluate  the  current  status  of  team  training  (Tz)  for  operators  of  complex  Air  Force  command  and 
control  (AFC2)  systems,  and  to  make  recommendations  for  enhancing  the  AFC2!’*  process.  The  research  was 
performed  for  the  Air  Force  Human  Resources  Laboratory  under  contract  F336I5-79-C-0025.  This  research 
effort  supports  a  major  new  Air  F’orce  Human  Resources  Laboratory  (AF'HRL)  research  and  development 
program  whose  primary  objective  is  to  improve  T2  technologies  in  areas  particularly  relevant  to  Air  Force 
combat  readiness.  The  program  objective  requires  the  establishment  of  a  baseline  data  base  on  how  T2  is 
currently  conducted  in  the  Air  F'orce.  and  how  it  is  developed,  implemented,  and  evaluated.  Because  Air 
Force  teams  vary  greatly  in  size,  structure,  and  functions,  it  would  be  impractical  to  collect  data  on  the 
training  provided  to  all  of  them.  Rather,  the  scope  of  this  research  effort  had  to  be  directed  at  an  area  with 
potential  high  payoff  for  increased  combat  readiness  and  effectiveness.  The  area  of  C2  was  chosen  as  a  point 
of  departure  for  the  research  because  CJ  teams  lend  to  lie  well  defined  structurallv.  are  of  a  manageable  size, 
and  perform  functions  highly  representative  of  Air  F'orce  mission  needs.  Furthermore,  as  the  research  effort 
unfolded,  limited  time  and  resources  made  it  necessary  to  focus  on  tactical  and  air  defense  C*  systems  to  the 
exclusion  of  strategic  C2  systems.  Thus,  the  C2  systems  surveyed  are.  or  in  the  case  of  planned  svstems  will 
become.  Tactical  Air  Command  (TAC)  resources. 

The  goal  of  this  effort  was  to  develop  a  picture,  through  interview  and  observation,  of  how  AF'C2T2  is 
currently  developed,  implemented,  and  evaluated,  and  what  C*  training  needs  will  arise  in  the  future.  Based 
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on  this  pit  tun*.  strengths  and  Hraknrs.scs  of  VFC^T*  won*  identified  ami  recommendations  were  developed 
in  three  areas: 

•  T*  research  and  development  program 

•  Resolution  o!  issues  using  current  lerhiii<pies/iechnulngies 

•  Simulation  teehnologv  development  for  (*‘T“ 

I  hese  recommendation'  will  form  tin*  foundation  for  future  research  bv  \FIIM  into  tin*  performance 
ol  (  J  trams  and  svsteni'.  1  In*  research  will  encompass  training  teehnologv .  performance  measurement 
let  hnitpies  lor  t  J  trams  and  svslems.  human  resources  issues  in  tin*  design  and  operation  of  (!2  svsteuis.  and 
training  of  rouiiiiaiid/dreiMun  skills.  The  ultimal<‘  :oal  of  tins  program  is  to  improve  leehnologies  in  areas  of 
tram  ami  human  factor*  related  to  thr  eoinhat  rflrctiveness  of  Air  Force  (l2  operations. 
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CHAPTER  I 


OVERVIEW  OF  THE  STUDY 

This  document  is  the  Executive  Summary  (Volume  V)  of  a  five- volume 

? 

report  on  a  survey  and  analysis  of  the  status  of  team  training  (T  )  Air 

2 

Force  systems  for  command  and  control  (C  )  and  technical  needs  to 
improve  training.  The  content  of  the  other  report  volumes  is: 

•  Volume  I:  The  Status  of  Air  Force  Team  Training  for 

2  2 

Command  and  Control  Systems  (AFC  T  ) 

2  2 

•  Volume  II:  Recommendations  for  an  AFC  T  Research 

Program 

•  Volume  III:  Recommendations  for  the  Application  of 

2  2 

Current  Technology  to  AFC  T 

2  2 

•  Volume  IV:  Recommendations  for  an  AFC  T  Simulation 

Facility 

2  2 

Data  on  Air  Force  team  training  for  command  and  control  systems  (AFC  T  ) 

were  collected  by  survey  and  observation  of  training.  The  status  of 
2  2 

AFC  T  was  then  characterized  in  terms  of  its  strengths  and  weaknesses. 
This  characterization  led  to  the  identification  of  issues  and  problem  areas 
which  could  be  resolved  through:  1)  immediate  application  of  available 
or  refineable  training  technology,  and  2)  a  more  extensive  research 
program.  Functional  requirements  were  derived  for  a  simulation  facility 
to  support  training  and  human  factors  research.  Recommendations  were 
made  for  the  pursuit  of  solutions  in  each  of  these  categories. 
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The  study  objectives,  approach,  findings,  conclusions,  and  recommendations 
are  summarized  in  the  following  pages. 

STATEMENT  OF  THE  PROBLEM 

2 

The  performance  and  training  of  teams  and  the  performance  of  C  systems 

are  being  increasingly  recognized  as  critical  military  problems.  The 

2 

problem  for  this  study  is  to  assess  the  adequacy  of  training  for  C  teams 

2 

in  the  Air  Force.  The  domain  of  AFC  teams  is  large,  and  the  focus  was 
2 

narrowed  to  C  teams  other  than  aircrews.  The  domain  was  further 

2 

narrowed  during  the  study  to  C  teams  in  Tactical  Air  Command  (TAC)  as 
a  scope  which  was  feasible  to  accomplish. 

2 

T  was  identified  as  a  major  need  for  military  systems  in  a  recent  report 
by  a  committee  of  the  Defense  Science  Board  (Reference  1),  and  a  high 
priority  research  program  was  recommended.  There  is  substantial 
literature  on  the  characteristics  and  operation  of  teams  and  groups  as 
indicated  by  recent  reviews  (References  2,  3,  4,  5,  and  6  are 
recommended). 

However,  the  amount  and  content  of  the  research  has  not  been  sufficient 
to  provide  a  systematic  body  of  knowledge  on  which  to  base  a  technology 
for  team  performance  and  training.  The  research  has  often  not  been 
focused  on  the  solution  of  problems  in  man- machine  operating  systems. 
However,  when  it  has  been  oriented  toward  these  problems,  the  laboratory 
tasks  and  constraints  have  been  seriously  unrepresentative  of  the  operational 
problems  and  conditions.  A  review  by  Denson  (Reference  7)  revealed  few 
findings  or  principles  which  would  allow  one  to  confidently  develop  and 


I 


implement  improved  training  for  Air  Force  teams.  This  lack  of  an  explicit 
2 

and  definitive  T  technology  leads  to  ineffective  and  inefficient  use  of  time 
and  resources. 

2 

The  seriousness  of  C  problems  have  also  been  given  visibility  by  reports 

and  pronouncements  following  the  Nifty  Nugget  War  Games  in  1978  (Reference 

8).  Performance  during  the  games  was  characterized  as  "plagued  with 

computer  foui  ups,  logistical  snarls  and....  'great  gaps'  in  understanding 

among  the  various  players.  "  Nifty  Nugget  was  regarded  as  the  climax  of 
2 

a  long  history  of  C  breakdowns.  The  plague  affected  all  levels  of 

command  from  individual  weapon  systems  to  theater  combat  commanders 

2 

to  the  National  Military  Command  Center.  The  core  of  the  C  problem 
was  identified  as  a  "complexity,  and  for  the  military,  complexity's 
not-so-distant  cousin,  chaos.  " 

An  interdisciplinary  group  under  Pentagon  sponsorship  was  put  to  the  task 
2 

of  analyzing  the  C  problem.  The  following  quotations  are  representative 
of  conclusions  by  that  group: 

"There  is  not  much  chance  for  the  formulation  of  a  command  and 
control  concept  so  long  as  the  dominant  paradigm  persists  in 
emphasizing  things  over  relationships  and  hardware  over  the  concepts 
needed  to  use  it  effectively.  This  narrow  point  of  view  needs  to  be 
shifted  if  the  fatal  consequences  surfaced  by  Nifty  Nugget  are  to 
be  avoided  if  and  when  our  forces  get  in  something  beyond  a  war  game.  " 


3 


"Much  of  the  expenditure  of  funds  in  command  and  control  at  present 
is  for  the  gathering,  communicating,  processing,  storage,  and  the 
display  of  information.  " 

"A  better  understanding  of  how  the  human  mind  uses  information, 
its  power  to  absorb  data  could  have  a  major  impact  on  technology 
requirements.  " 

These  conclusions  indicate  that  the  people- system  interface  is  a  significant 

O 

contributor  to  the  C“  problems. 

The  study  reported  herein  was  in  support  of  the  AFHRL  objective  in  the 
2 

area  of  advancing  T“  technology.  The  first  necessary  step  in  order  to 

ensure  relevance  to  Air  Force  mission  needs  was  the  establishment  of  a 

2 

baseline  data  base  on  how  AFT  is  currently  developed,  implemented,  and 
evaluated.  Because  Air  Force  teams  vary  greatly  in  size,  structure,  and 
functions,  it  would  have  been  impractical  to  collect  data  on  the  training 
provided  to  all  of  them.  Rather,  the  scope  of  this  research  effort  had  to 
be  directed  at  an  area  with  potential  high  payoff  for  increased  operational 
readiness  and  effectiveness. 

2 

The  area  of  C  was  chosen  as  a  point  of  departure  for  the  research  because 
2 

C  teams  tend  to  be  well  defined  structurally,  are  of  manageable  size,  and 

perform  functions  highly  representative  of  Air  Force  mission  needs. 

Furthermore,  as  the  research  effort  unfolded,  limited  time  and  resources 

2 

made  it  necessary  to  focus  on  tactical  and  air  defense  C“  of  air  resources 

2 

to  the  exclusion  of  strategic  C  .  This  limitation  does  not  affect  the  usefui- 

2  2 

ness  of  the  research,  because  assessing  tactical  and  air  defense  C  T 


necessitated  addressing  issues  of  T  for  performance  in  emergent  (that  is, 
unpredictable),  high-stress  environments,  as  well  as  in  established, 
highly  proceduralized  environments. 

The  basis  for  the  decisions  to  exclude  strategic  systems  was  a  difference 

2 

in  the  performance  requirements  for  strategic  and  tactical  C  systems  and 

teams.  They  differ  along  three  situational  dimensions  summarized  in 

2 

Table  1.  Strategic  C  is  characterized  by  specifiable  environmental 

conditions,  predictable  system  states,  and  available  solutions  and  probable 

2 

consequences.  On  the  other  hand,  tactical  C  is  characterized  by  unspec- 

ifiable  conditions,  unpredictable  system  states,  and  less  available  options 

and  probable  consequences.  This  difference  suggests  that  strategic  and 
2 

tactical  C  teams  are  different  in  structure  and  performance  requirements 
and  thus  have  different  training  needs.  The  informal  data  regarding 

TABLE  1.  CHARACTERISTICS  OF  STRATEGIC  AND  TACTICAL  C2 


Strategic 

Tactical 

Environmental 

Conditions 

Specifiable 

Not  specifiable 

States  of  the 

System 

Predictable 

Not  predictable 

Probable 

Ava  ilable 

Potentially 

Consequences 

(Solutions) 

available 

IP 


2 

strategic  C  collected  during  this  effort  suggest  that  this  is  an  accurate 

statement.  The  more  emergent  or  unexpected  nature  of  the  tactical  situa- 

2 

tion  indicates  that  tactical  T  would  be  more  difficult  to  train.  Training 
must  produce  some  skills  that  are  generalizeable  to  a  large  set  of  possible 
conditions,  states,  options,  and  consequences.  The  difficulty  entails 
producing  those  skills  in  such  a  way  that  combat  readiness  can  confidently 
be  assumed. 

2 

Improvements  in  T  can  be  expected  to  have  a  higher  payoff  in  tactical 

9 

C“  systems  owing  to  this  emergent  nature  of  tactical  situations  and  the 

emphasis  on  real-time  decision  making.  Therefore,  it  was  decided  to 

2 

concentrate  on  tactical  C  systems. 

The  objectives  of  this  study  were  to: 

2  2 

•  Develop  a  data  base  on  how  .AFC  T  is  currently  defined,  developed, 
implemented,  and  evaluated  (Volume  I,  Chapters  II,  III,  and  IV). 

2  2 

•  Identify  strengths  and  weaknesses  of  AFC  T  and  identify  high 
payoff  issues  and  problem  areas  (Volume  I,  Chapter  VI). 

•  Determine  which  problem  areas  require  further  research  and 
recommend  areas  best  addressed  through  a  program  of  research 
(Volume  H). 

•  Determine  which  issues  or  problem  areas  could  be  addressed 
through  currently  available  or  refineable  technology,  and 
recommend  topics  for  immediate  application  (Volume  III). 

•  Determine  the  functional  characteristics  required  in  a  simulation 

2  2 

facility  to  support  C  T  research  and  development  (Volume  IV). 


6 


STUDY  APPROACH 


On-site  data  collection  and  documentation  review  were  accomplished  in 

2  2 

order  to  develop  a  data  base  descriptive  of  the  characteristics  of  AFC  T  . 
Interviews  and  observations  were  the  primary  survey  methods,  supplemented 
by  reviews  of  appropriate  technical  material. 

Interviews  were  conducted  with  personnel  associated  with  training  for  the 

Tactical  Air  Control  System  (TACS),  Tactical  Air  Control  Center  (TACC), 

Control  and  Reporting  Center  (CRC),  the  Airborne  Warning  and  Control 

System  (AWACS),  and  the  Semi- Automated  Ground  Environment  (SAGE) 

system.  Personnel  included  program  managers,  training  developers, 

2 

instructors,  students,  and  evaluators.  In  addition,  personnel  in  C  system 
design,  development,  and  procurement  at  the  Electronic  Systems  Division 
(ESD)  and  MITRE  Corporation  of  Air  Force  Systems  Command  (AFSC) 
were  interviewed. 

All  interviews  were  carried  out  with  specially  designed  guides,  each 

oriented  towards  the  particular  type  of  respondent;  for  example,  different 

interview  guides  were  created  and  used  for  instructors,  students,  and 

so  on.  The  questions  asked  were  designed  to  obtain  information  about  how 
2 

T  was  designed,  implemented,  and  evaluated. 

Observations  of  training  and  training  devices /simulators  were  also  carried 
out  according  to  guides  developed  especially  for  this  effort.  The  interview 
and  observation  guides  can  be  found  in  Appendix  B  of  Volume  I. 
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The  interviews  and  observations  were  supplemented  with  a  review  of 
2 

documents  on  AFC  systems  and  technical  literature  on  Air  Force  teams 
2 

and  T  when  the  review  by  Denson  (Reference  7)  was  done. 

2 

Air  Force  C  systems  are  complex,  organized  entities  composed  of 

many  elements.  Figure  1  depicts  the  Air  Force  system  which  supports 

joint  force  operations.  (The  acronyms  in  the  figure  are  decoded  in  Table  2.  ) 

Each  center,  post,  unit,  or  aircraft  depicted  in  Figure  1  is  itself  a  system, 

or  more  properly,  a  subsystem  of  the  tactical  operations  structure.  Each 

o 

subsystem  consists  of  personnel  equipment  organized  to  allow  certain  C~ 
functions  and  tasks  to  be  carried  out. 

The  CRC  of  the  TACS,  for  example,  typically  consists  of  approximately 

90  people,  a  hardware /software  system  known  as  the  TSQ-91  (407L),  and 

other  equipment.  It  is  organized  into  roughly  four  functional  areas.  Two 

areas,  surveillance  and  identification,  are  devoted  to  the  acquisition  of 

information;  command/ battle  staff  does  decision  making;  and  the  weapons 

2 

control  area  directs  implementation  of  tactical  action.  This  type  of  C 
team  structure  is  illustrated  in  Figure  2.  Each  of  these  functions  is 
performed  by  a  number  of  personnel  organized  in  a  section  supported  by 
equipment  or  information.  The  essential  components  of  a  TSQ-91  are 
consoles  which  display  radar  imagery  and  the  personnel  who  observe  it  to 
determine  the  location,  heading,  and  speed  of  aircraft  and  missiles. 
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Figure  1.  Tactical  Air  Force  Supporting  Joint  Force  Operations 


ABCCC 

Airborne  Command  and  Control  Center 

AFCH 

Air  Force  Component  Headquarters 

ALCC 

Airlift  Command  and  Control 

ALCE 

Airlift  Control  Element 

ASOC 

Air  Support  Operations  Center  (formerly  DASC--Direct 

Air  Support  Center) 

ASRT 

Air  Support  Radar  Team 

BDE 

Brigade 

BN 

Battalion 

CCT 

Combat  Control  Teams 

CRC 

Control  and  Reporting  Center 

CRP 

Control  and  Reporting  Post 

DIV 

Division 

FAC 

Forward  Air  Controller 

FACP 

Forward  Air  Control  Post 

JCS 

Joint  Chiefs  of  Staff 

JOC 

Joint  Operations  Command 

NCA 

National  Command  Authority 

SCAR 

Strike  Control  and  Armed  Reconnaissance 

T  ACC 

Tactical  Air  Control  Center 

TACP 

Tactical  Air  Control  Party 

woe 

Wing  Operations  Center  (formerly  TUOC--Tactical 

Unit  Operations  Center) 
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Figure  2.  Generic  Structure  of  Air  Force  Tactical 

or  Air  Defense  Command  and  Control  System 


A  team  of  performance/training  models  was  adopted  which  consists  of 
three  dimensions: 

•  Performance  context 

•  Personnel  category 

•  Training  program  type 
The  model  is  depicted  in  Figure  3. 

2 

The  performance  context  is  elaborated  further  by  application  to  T“  for  a 
weapons  controller  in  a  CRC/CRP.  The  results  of  that  analysis  are 
presented  in  Figure  4.  It  is  based  on  a  concept  proposed  by  Brock 
(Reference  9). 

FINDINGS 

The  data  obtained  through  the  survey  were  analyzed  in  terms  of  training 

program  definition  and  development  (Volume  I,  Chapter  II),  implementation 

(Volume  I,  Chapter  III),  and  evaluation  (Volume  I,  Chapter  IV).  Figure  3 

shows  the  three  primary  dimensions- -performance  context,  personnel 

o  2 

category,  and  training  program  type--into  which  AFC“T  can  be  structured. 
Each  cell  of  the  matrix  can  in  principle  be  addressed  by  training  develop¬ 
ment,  implementation,  and  evaluation. 

Table  3  summarizes  the  strengths  and  Table  4  the  weaknesses  found  by 
2  2 

this  survey  of  AFC  T  .  There  are  more  weaknesses  than  strengths,  but 
the  weaknesses  tend  to  be  symptoms  of  a  relatively  small  number  of  under¬ 
lying  issues  and  problem  areas  which  are  discussed  in  the  Conclusions 
section. 
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EXERCISES 
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b’igure  3.  Training  Contexts  to  which  Instructional 
System  Development  May  He  Applied 


gure  4.  Team  Training 


awareness  and  a  shared  plan. 


TABLE  4.  WEAKNESSES  OF  AFC2T2 


Definition  and  Development 

Implementation  and  \lanag*>m»  m  i con* i  <ued> 

Enforcement  oi  human  factors  data  requirements  is 
incomplete  or  lax  as  a  result  of  pressures  to  develop 
the  system. 

Training  is  often  viewed  as  a  panacea  for  ineffective 
system  design  which  it  is  not. 

Joint  services  svstem  acquisition,  for  example. 

TRl-TAC,  put  stress  on  training  developers  who 
might  have  to  contend  with  different  ’raining 
philosophies  -ind  personnel  constraints. 

In  competition  for  scarce  or  limited  resources  the  C^t2 
area  has  historically  had  a  low  priority  m  the  Air  Force. 

There  3 re  no  systematic  procedures  for  defining  the 
appropriate  team  structure  tor  a  system  or  for 

allocating  tasks  to  the  team  as  opposed  to  the 
individual  operator  or  svstem  software. 

Instructional  System  Development  <ISD«  techniques  do 
not  address  team  tasxs  or  stalls  or  T-  objectives,  nor 
do  they  adequately  address  non- console  rasxs  and  the 
tasxs  of  non- operators. 

There  are  no  systematic  procedures  tor  defining 
svstem  simulator  requirements;  nor  is  there  empirical 
data  on  the  level  of  fidelity  required  for  C”T^. 

The  procurement  ot  C-  systems  radically  different 
from  existing  systems,  for  example,  AW. ACS,  puts 
pressure  on  training  developers  because  job  ' 
experts  find  it  difficult  to  relate  to  unfamiliar 
operating  procedures. 

The  definition  of  T^  requirements  is  hampered  by  a 

LacK  ot'  articulation  of  what  proficient  C-  team 
performance  consists  of. 

Training  requirements  for  operations  framing  programs 
in- unit i  are  oi  dubious  validity  because  thev  have  not 
been  related  empirically  to  C-  skills  achievement  and 
maintenance. 

Operations  training  program  requirements  for  live  T“ 
are  constrained  bv  the  availability  of  flying  resources. 

Training  objectives  are  developed  informally,  at  ail. 

for  transition  and  continuation  training  in  the  'ontext 
of  simulated  combat  missions,  svstem  exercises. 

The  selection  of  instructional  methods,  media,  and 
sequencing  is  primarily  influenced  bv  factors  unrelated 
to  training  effectiveness;  nameiy,  tradition,  resource 
constraints,  etc. 

'  aified,  comprehensive  plans  dealing  with  the  manage¬ 
ment  and  production  of  operations  training  do  not  exist. 

Implementation  and  Management 

There  ire  no  minimum  aptitude  requirements  for  entry 
.nto  the  Ur  Weapons  Controller  career  field. 

Team-oriented  skills  are  rot  trained  systematically; 
sequencing  of  training  :s  not  optimized. 

2 

There  ;s  no  formal,  standardized  training  for  C  -vstem 
supervisors. 

The  emergent  nature  of  tactical  missions  maxes  tactical 
skill  training  dtfficui’:  simulated  crr.bat  missions  -v«t**m 
exercises  for  \WAO  and  T  U‘?  are  :ii  d«  urn-d. 

The  evaluation  of  team  readmes.-  .s  hampered  bv  i  s  . 

articulation  of  the  dimensions  .»ud  ittrib;te«*  :  *ood  team 

P*’i  .crmano-. 

Team  readiness  assessment  .s  jnstanna rdir-  d  .n  t.  rtr.s  y 
conditions  and  iifficula. 

The  importance  of  f^edbacx  'o  ’earns  .<*  r*  •  ogmz*»d  1  u* 
there  ar*-  c»o  proven  •» •cj.iuqjes  for  maku.g  »  dta.  s  ■  '.*•  .  •..*■. 

Simulators  do  .not  fuliv  suppor’  .tistru*  *or  requ.  rumen's. 

Simulation  f.deutv  does  r.ot  .-.upport  effective  Tlif.i:  g. 

Personnel  ajio  simulate  ;n?«  r  eptor  punts  are  >r  r  -r v 
teamed  for  ’heir  duty. 

*> 

C"  career  fields  are  c:iar:*t  tm? •  d  bv  .ow  . . it  - 

and  the  steaclv  loss  of  ••xperiem  .-d  .ndi.idjais.  1 

C  "  career  fields  are  characterized  bv  :ow  ntc.ti  r..t.  -  J 

and  the  steadv  loss  of  expe-i*  j  >  •  -t  .r.d; vidua.1*.  1 

C-  training  program  managers  have  an  .nremrie*.  r*a*  r 
standing  of  the  t  - T”  pipeline. 

Evaluation  ..r.d  Modification 

The  sue  ••  ssiui  i>e  *>f  an  LSD  malvs.s  *o  ••vai.;at«  u  ’ram.nj 
program  iepends  on  nav»”g  unbiased  -ubi-  .  *  •  u'te; 
experts;  because  instructors  .n  ’he  program  .-uaiiv 
provide  ••xp*  rtise  1  hen*  is  inherent  wa.-  .u  ■  L«  • 

•'valuation. 

Th«  lacs.  of  obi»»<  uve,  behavior- refer#  need  *r:t»  ria  for 
assessing  *ea;i.  oriented  skiIIs  and  :-Jm  performance 

Lin. its  the  star  e«.-al  effort. 

O.'-en  the  unvaudatrd  reiavionship  between  operations 
training  program  requirements,  in  '"r:v.«  oi ,  for  example, 
’lumber  of  intercepts  a  quarter,  and  *kiIU  maintenance 
the  achievement  of  ‘hose  requirements  should  not  b.*  used 
as  an  eva.uative  measure . 

The  measurement  tools  of  *he  exoert  are  unsoprustu  iteri, 
informal,  jr.docjim  nt«*d.  and  depend  on  the  experien.  e  u 
the  particular  ••xper*  involved. 

Incomplete  documentation  of  ISO  maivses  complicates 
program  modificai .on  jivi  muKe-  .t  :n»v'r  .  .ent. 

Training  Programs  ire  •><  cnsionailv  modified  <s  a 
of  rt-i<ourc<-  -  lortage.s  or  ■  nang»  s  4u  mui.ugemem  nr...  •**orb\ 
and  such  modit;.  ations  fr«  quentn  reduce  *he  iualit\  ot 
’raining. 

There  are  few  resource*  and  .vr  low  prior;*'  for 
modifying  simulators  wruh  ir**  cnown  or  demonstrated  *o 
be  deficient  m  fidelity  or  •  aoabilitv. 
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Strengths 


Throughout  the  survey  it  was  evident  that  AFC  training  personnei  were 

2 

concerned  about  improving  T  effectiveness.  The  picture  that  emerged  was 

of  competent,  dedicated  individuals  doing  the  best  they  could  with  limited 

2  2 

resources  and  technologies.  There  is  recognition  of  the  weakness  of  C  T 
and  a  desire  to  take  positive,  corrective  action. 

Training  for  individual  operators  is  effectively  developed  and  implemented. 
The  assessment  of  operators'  individual  skills  is  comprehensive  and  well 
standardized.  Good  efforts,  for  example,  intrateam  briefings,  are  made 
at  building  team  identity  and  spirit. 

Weaknesses 


2  2 

The  survey  found  a  lack  of  formal  T  programs  for  AFC  systems.  The 

system  and  large-scale  exercises  that  do  exist  have  tended  to  become 

ends  in  themselves  rather  than  being  based  on  and  targeted  to  the  job 

requirements  of  an  operational  environment.  Exercises  are  limited  by 

availability  of  live  flying  resources  and  inadequate  simulation  capabilities 
2 

to  support  T  . 

It  was  also  found  that  current  measurement  techniques  do  not  support  team 
or  training  program  evaluation.  The  lack  of  diagnostic  measures  of  team 
operational  readiness  will  hamper  research  and  application  efforts  aimed 
at  increasing  it. 
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A  critical  problem  exists  in  the  C  career  fields.  Respondents  to  the 
interview  attributed  it  to  low  job  satisfaction  and  dissatisfaction  with 
opportunities  in  the  career  field.  Great  pressure  is  exerted  on  the  training 
pipeline  to  provide  proficient  replacements.  Given  the  importance  of 
subject  matter  experts  throughout  the  training  cycle,  the  rapid  turnover 
and  loss  of  experienced  individuals  threatens  the  foundations  of  effective 
.Air  Force  operations. 

The  low  priority  given  to  human  factors  and  training  in  system  acquisition 

exacerbates  the  problems  that  already  exist  and  causes  others.  Poor 

system  design  can  sometimes  be  overcome  through  training,  but  this  is 

neither  desirable  nor  does  it  make  effective  use  of  training  resources.  A 

limited  number  of  actual  equipment  trainers  are  available,  and  they  have 

limited  capabilities  to  support  training,  especially  in  the  areas  of  training 

functions  to  support  student  aid  instructors.  This  deficiency  is  a  result 

of  limited  resources  for  training,  unavailability  of  knowledge  to  address 
2 

T  requirements  to  procure  simulators  and  training  devices,  and  inadequate 
2 

provision  for  T  requirements  in  the  design  and  procurement  phases  for 
2 

C  systems. 

ISSUES  AND  PROBLEM  AREAS  IN  AFC2T2 

2  2 

The  weaknesses  identified  in  the  survey  of  AFC  T  were  consolidated  into 
issues  and  problem  areas.  Table  5  contains  the  results  of  this  consolida¬ 
tion  organized  into  the  categories  of  definition  and  development,  implementa¬ 
tion  and  evaluation  of  training  programs,  and  personnel  policy  and  resource 
constraints.  Personnel  policies  and  resource  constraints  are  not  typically 
matters  for  research  and  development,  but  they  have  a  major  impact. 
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instiiut ional  methods  and  sequencing  for 
subteam,  team,  and  superteam  skill  training 


often  negative.  There  may  be  little  that  can  be  done  about  them  through 
research  or  technological  applications  in  human  factors  and  training. 
Studies  can  be  done  to  generate  information  on  the  effect  or  causes  of  a 
shortage  of  live  flying  events,  loss  of  experienced  personnel,  and  poor 
understanding  of  C“  training  by  program  managers.  But  changing  those 
conditions  and  constraints  is  difficult  because  the  policies,  allocations  of 
resources,  and  constraints  are  the  responsibility  of  persons  in  other  areas 
who  must  also  weigh  other  priorities.  One  can  develop  and  recommend 
alternative  man-machine  configurations  and  manpower  practices  to 
alleviate  the  effects  of  the  constraints.  One  can  also  use  research  results 
to  recommend  and  lobby  for  changes,  but  the  decision  falls  under  the 
charter  of  someone  in  another  chain  of  command. 

A  brief  discussion  of  each  issue  or  problem  follows.  For  a  detailed 
discussion,  see  Volume  I,  Chapter  V. 


Lack  of  a  Definitive  Framework  for  Analyzing  Team 
Skills  and  Designing  Team  Structure  and  Function 


The  Research  and  Development  community  has  failed  to  deal  comprehensively 
and  consensually  with  this  fundamental  issue.  This  failure  derives 
primarily  from  the  lack  of  a  body  of  universally  accepted  representative 
data  on  teams  and  a  lack  of  a  correponding  team  performance  theory; 
no  science  of  team  performance  exists. 


A  solution  to  this  issue  entails  the  development  of  a  conceptual  framework 

embracing  both  a  taxonomy  of  team-oriented  skills  and  a  methodology  for 

2 

considering  team  characteristics  in  C  system  design.  The  characteristics 
2 

which  a  T  conceptual  framework  should  possess  are: 
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2 

1.  A  classification  scheme  of  types  of  C  teams  according  to 
structure  functions,  missions,  etc. 

2.  _4n  identification  of  the  dimensions  of  the  process  of  team 
performance  and  variables  associated  with  task  output 

3.  A  taxonomy  of  team-oriented  skills  along  with  a  task  analytic 
procedure  which  forces  attention  to  cognitive  skills  necessary 
in  cooperation,  coordination,  and  communication 

9 

The  characteristics  of  a  C  system  design  framework  should  include: 

1.  Guidelines  for  determining  mission/ task  functions  allocatable 
to  machines,  individuals,  and  teams 

2.  Guidelines  which  efficiently  support  the  functional  allocation 
of  tasks 

3.  Information  predictive  of  the  effects  of  task  or  position 

2 

automation  in  C  systems 

Lack  of  Objective  Criteria  and  Standards  for  Evaluating  Team-Oriented 

Skills.  Individual  and  Team  Readiness,  and  Cz  System  Effectiveness 

No  measurement  tools  exist  to:  1)  assess  individual  team-oriented  skills 

and  relate  their  level  to  some  standard  (of  readiness),  2)  assess  team 

2 

readiness,  or  3)  evaluate  C  system  effectiveness. 

Resolving  the  problem  of  measuring  an  individual's  team-oriented  skill 

proficiency  requires  development  of  a  valid  task  taxonomy  as  discussed 

above.  The  standards  of  task  performance  can  be  easily  described  in 
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established  situations  where  responsibilities  and  operating  procedures 
are  defined.  But  the  absence  of  standards  is  part  of  the  definition  of 
emergent  situations  which  characterize  tactical  engagements;  there  may  be 
more  than  one  acceptable  solution  to  a  problem.  This,  in  fact,  is  the 
primary  reason  for  the  importance  in  individual  evaluation  by  observers  who 
are  expert  in  the  performance  under  observation.  Better  articulation  of 
the  standards  required  of  individual  team-oriented  skills  is  needed- -one 
taking  into  account  the  unpredictable  nature  of  tactical  engagements. 

Observation  of  the  process  of  performance  should  be  at  the  heart  of 
evaluating  team  readiness  and  system  effectiveness.  Readiness  cannot  be 
judged  solely  by  concrete  measures  of  number  of  exercises,  activities 
engaged  in  per  unit  time,  or  percent  of  targets  detected/intercepted. 

Such  information  is  convenient  for  obtaining  summary  statistics  for  use  by 
higher  headquarters.  However,  it  overlooks  critical  factors  such  as  the 
quality  or  difficulty  of  the  experiences  or  the  process  of  achieving  mission 
objectives.  Such  process  information  is  captured  to  some  degree  in  after¬ 
action  reports,  but  these  are  not  used  to  full  advantage.  Part  of  the  reason 
they  are  not  used,  especially  for  program  evaluation  and  by  higher  head¬ 
quarters,  is  because  of  their  narrative  format. 

.Any  approach  to  resolving  these  related  problems  must  focus  on  improving 

the  tools  of  the  subject  matter  expert  who  is  still  the  only  valid  source  of 

2  2 

evaluative  information  in  .AFC  T  .  Resolution  of  these  problems  is  a 

2 

prerequisite  to  conducting  useful  T  research;  it  is  difficult  without 
resolution  to  ensure  standards  for  quality  control  in  training  and  operational 
readiness. 
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Lack  of  Analytic  Techniques  and  Empirical  Data  for  Determining 
Institutional  and  Operational  Requirements  and  Objectives 

2 

The  definition  of  T  requirements  and  objectives  is  essential  in  order  that 

training  be  carried  out  or  improved  in  a  consistent,  verifiable  manner. 

This  issue  underlies  a  number  of  weaknesses  in  the  definition  and  develop- 

2 

ment  of  training  during  C  system  acquisition  and  for  operations  training 
programs. 

Joint  service  acquisitions,  the  training  of  experienced  individuals  for  new 
systems  initial  deployment,  and  systems  which  are  a  clear  departure  in 
design  and/or  function  from  existing  systems  ail  create  especially  trouble¬ 
some  problems  for  Air  Force  training  developers.  This  is  true  even  for 
straightforward  console  operator  training.  The  problems  are  magnified 
when  non-console,  non-procedural  tasks  are  concerned. 

Ideally,  the  instructional  system  development  (ISD)  framework  should 

provide  procedures  and  techniques  for  dealing  with  these  situations.  It  does 

not.  ISD  works  well  in  highly  structured,  predictable  tasks  environments, 

but  tends  to  become  ineffective  when  the  process  of  performance  rather 

than  its  outcome  is  of  importance.  Because  process  is  the  critical  feature 

of  team  performance  in  emergent  situations,  ISD  in  its  present  form  is  not 

2 

a  useful  tool  for  developing  T  . 

Although  it  is  important  to  develop  systematic  procedures  for  establishing 
2  2 

initial  C  T  knowledge  and  skill  requirements,  it  is  equally  if  not  more 

2 

important  to  develop  a  data  base  regarding  the  forgetting  of  C  skills. 
Without  such  data,  it  is  difficult  to  validly  define  the  requirements  for 

; 

i 


refresher  training  and  operational  skills  maintenance.  The  current 
practice  of  requiring  a  certain  number  of  system  exercises  and  events, 
or  hours  of  activities  per  month  or  quarter,  is  of  dubious  validity  because 
there  are  no  empirical  data  establishing  the  relationship  between  performance 
frequency  and  combat  readiness.  Such  data  are  needed. 

Lack  of  Comprehensive,  Systematic  Procedures  for  Defining 
Training  Objectives  for  Simulated  Combat  Missions 

Simulated  combat  missions  are  utilized  for  exercising  individual  positional 

and  team-oriented  skills.  This  technique  is  necessary  for  achieving  and 

2 

maintaining  operational  readiness.  Its  major  function  is  to  expose  C 
teams  to  the  quantity  and  quality  of  events  which  they  might  face  in  actual 
combat.  The  specification  of  the  quantity  and  quality  of  these  events  is  the 
subject  of  this  issue.  There  is  currently  no  set  of  documented,  standard 
guidelines  for  defining  system  exercise  training  objectives,  especially 
when  the  system  mission  deals  with  emergent  situations. 

The  definition  of  training  objectives  for  system  exercises  is  complicated 
by  a  number  of  factors.  It  first  requires  the  expertise  in  military  tactics 
and  doctrine  and  additional  knowledge  which  comes  primarily  with  combat 
experience.  Next  it  is  necessary  to  create  plausible,  meaningful  scenarios 
of  events  for  missions  which  must  capture  the  emergent  nature  of  tactical 
engagements.  This  requires  a  great  deal  of  imagination  and  must  be  tied 
to  the  variability  of  mission  requirements  which  characterize  a  world-wide 
military  potential  (TACS  and  AWACS).  Furthermore,  the  quality  and 
quantity  of  events  must  be  tied  to  some  index  of  training  value  either  in 
terms  of  team  skills  or  mission  task  skills  or  both. 
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These  factors  must  be  accommodated  by  any  procedures  or  guidelines 
developed  for  defining  training  objectives  for  simulated  combat  missions 


Inadequate  Planning  and  Analytic  Techniques  for  Defining 
T^  Simulation  Fidelity  and  Functional  Requirements 

This  issue  reflects  the  fact  that  the  fidelity  and  capabilities  of  current 

simulations /simulators  are  more  related  to  the  level  of  technology  available 

or  acquisition  resource  limitations  than  to  what  might  be  required  to 

suDport  training.  Resolution  of  this  issue  entails  the  development  of  a  data 

base  relating  simulation  characteristics,  cost,  and  performance  benefit. 

Failure  to  resolve  this  issue  will  undermine  the  critical  and  expanding  role 

2  2 

that  simulation  must  play  in  C  T  . 

Failure  to  Define  and  Develop  Formal  Training  for 
C2  System  Supervisors  ~ 

This  discussion  applies  to  the  implementation  issue  of  lack  of  instruction 
for  supervisors,  battle  staff  personnel,  and  decision  makers.  This  issue 
is  to  a  large  degree  a  matter  of  policy,  not  the  consequence  of  a  lack  of 
applicable  training  technology.  Strong,  competent  leaders  who  thoroughly 
understand  their  jobs  are  a  prerequisite  for  superior  team  performance. 
Yet,  in  the  Air  Force,  system- specific  training  for  supervisors /managers 
is  largely  on-the-job  training  (OJT)  during  simulated  missions;  it  is 
informal  and  unsystematic.  These  conditions  are  not  usually  sufficient 
for  effective  training.  To  make  more  effective  use  of  supervisor  OJT 
experiences,  they  must  be  structured  and  consistent  and  designed  according 
to  systematically  defined  training  objectives.  If  this  were  done,  training 
time  to  operationally  ready  status  could  be  reduced  substantially. 
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Deficient  Simulation  Capabilities 

2  "> 

The  accepted  method  of  stimulation  in  AFC  T“  is  stimulation  of  actual 
radar  display  equipment.  Actual  equipment  trainers  (AETs)  have  historically 
had  three  shortcomings:  1)  It  is  costly  to  develop  stimulation  materials  to 
generate  realistic  synthetic  imagery;  2)  Some  display  conditions  which  are 
desirable  if  not  "musts"  for  training,  are  too  difficult  technically  and  too 
costly;  and  3)  Instructional  features  and  trainer- instruction  interface 
are  meager  if  provided  at  all.  These  features  are  means  for  controlling 
stimuli,  difficulty  of  problems,  and  measurement  of  performance,  for 
example.  In  initial  training  for  air  weapons  controllers,  AETs  have  two 
major  weaknesses.  First,  they  fail  to  accurately  represent  aircraft  radar 
tracks  in  terms  of  speed,  rate  of  turn,  and  so  on.  Second,  AETs  inadequately 
support  the  instructor  in  such  areas  as  performance  recording/management 
and  record  keeping. 

The  problems  with  AETs  are  magnified  in  operations  training  during 
simulated  combat  missions  and  system  exercises.  In  this  context,  in 
addition  to  the  above  weaknesses,  AETs  fail  to:  2)  eliminate  the  tracks  of 
targets  which  have  been  destroyed,  2)  provide  the  level  of  target  density 
expected  in  a  (European)  theatre  operation,  3)  simulate  fully  the  effects  of 
electronic  warfare,  including  the  use  of  el.  ctronic  warfare,  including  the 
use  of  electronic  countermeasures  and  counter- countermeasures,  and  4) 
simulate  sensor  management. 

These  deficiencies  are  acknowledged  by  Air  Force  training  personnel. 

Solutions  to  some  of  the  problems  for  the  407L,  the  nomenclature  for  the 

CRC  equipment  complex,  will  be  obtained  through  the  deployment  of  the 

System  Training  Exercise  Module  (STEM).  The  STEM  will  replace  the 
2  ,  4 

current  T  /  T  system. 
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There  is  a  lack  of  hard  data  concerning  the  training  effectiveness  of  AETs. 
Consequently,  it  is  difficult  to  know  in  which  areas  the  upgrade  of  capabilities 
will  be  cost  effective.  Instructor  support  capabilities  could  be  added 
confidently  with  the  knowledge  that  training  effectiveness  would  be  improved. 
Other  enhancements,  for  example,  in  the  area  of  electronic  warfare 
simulation,  would  be  costly.  However,  it  is  most  important  that  development 
of  any  of  the  enhancements  is  initiated  by  an  analysis  of  the  job  and  associated 
tasks  in  order  to  derive  requirements  for  simulation  which  provide  sufficient 
fidelity  and  are  responsive  to  the  performance  requirements  of  the  job. 

Mismatch  Between  Entry  Level  Requirements  and  Air 
Weapons  Controller  Fundamentals  Course  Syllabus 

There  are  no  specific  entry  level  aptitude  requirements  for  the  air  weapons 
controller  career  field  (17xx)  beyond  the  requirements  for  the  officer  ranks. 
Some  respondents  in  the  survey  stated  that  successful  air  weapons  controllers 
(AWCs)  need  skill  in  mathematics,  spatial  reasoning,  and  communication. 
They  further  indicated  that  deficiencies  in  these  skills  are  characteristic 
weaknesses  of  AWC  Fundamentals  Course  students.  (We  could  not  determine 
whether  these  deficiencies  did,  in  fact,  impair  operational  performance. 

Our  evidence  is  hearsay  at  best.  )  However,  this  problem  places  a  burden  on 
all  subsequent  units  in  the  controller  training  pipeline  and  has  obvious 
consequences  for  the  quality  of  personnel  in  the  career  field. 

This  issue  must  be  considered  in  light  of  .Air  Force  manpower  resources 
and  required  manning  levels.  At  the  present  time  the  17xx  career  field 
is  critically  undermanned  and  projections  indicate  this  situation  will  get 
no  better.  Thus,  although  psychometric  techniques  are  available  for 


developing  selection  criteria,  it  is  unclear  whether  such  criteria  could 
be  applied  to  the  available  officer  candidate  population.  In  view  of  a  demand 
greater  than  supply,  the  Air  Force  will  not  be  able  to  be  selective  until 
the  balance  shifts.  The  mismatch  between  what  is  taught  and  what  is 
required  given  the  qualifications  of  the  students  will  continue  to  exist  until 
corrective  action  is  taken.  Meanwhile,  data  can  be  generated  to  structure 
remedial  training  programs  or  influence  management  policy  if  an  actual 
case  can  be  documented. 


Lack  of  Empirical  Data  Regarding  the  Optimal  Instructional  Methods 
and  Sequencing  for  Subteam,  Team,  and  Superteam  Training 


o 

A  systematic  training  methodology  for  T“  does  not  exist.  Systematic 

approaches  to  training  presume  a  building  block  approach  in  which 

components  of  knowledge  and  skill  are  integrated  to  provide  behavioral 

capabilities  needed  on  the  job.  The  course  of  training  is  a  progression 

through  intermediate  behavioral  objectives  that  increasingly  approximate 

terminal  performance  objectives.  This  systematic  shaping,  however, 

2 

does  not  occur  in  T  .  There  are  two  reasons  for  this.  First,  team 

knowledge  and  skill  objectives  are  defined  only  in  broad  general  terms. 

The  objectives  are  not  analytic  enough  to  support  sequencing  or  selecting 

particular  training  methods  which  might  be  superior  to  others.  Second, 

existing  training  methods  might  not  support  the  needs.  In  fact,  there  is 

2 

a  need  for  data  regarding  the  success  of  various  methods  in  T  . 


Lack  of  Training  for  Support  Personnel 
Who  Simulate  Interceptor  Pilots 


This  issue  is  prominent  in  initial  and  initial  transition  training  programs, 
specifically  the  Air  Weapons  Controller  Fundamentals,  Automatic 
Positionally  Qualified,  and  TACS  407L  courses.  The  individuals  who 
simulate  interceptor  pilots--interceptor  pilot  simulators  (IPS)--do  not 
receive  any  training  beyond  orientation  to  the  equipment  and  radio  vocabulary 
pertinent  to  the  task  they  perform.  As  a  result  the  training  situation  for 
AWCs  is  suboptimal. 

A  related  problem  is  that  during  live  flying  in  these  same  courses,  the 
pilots  are  usually  students  themselves.  Because  their  skills  are  not 
necessarily  sharp,  the  training  experience  for  both  parties  can  leave  much 
to  be  desired.  For  example,  the  student  pilot  may  not  respond  correctly 
or  as  quickly  as  an  experienced  pilot  to  controller  instructions.  Student 
AWCs  may  learn  to  compensate  for  these  errors  and  may  thus  learn 
techniques  inappropriate  to  communicating  with  experienced  pilots. 

Lack  of  Valid  Measures  in  Program  Evaluation 

Currently,  questionnaires  are  typically  used  for  evaluation  of  institutional 
training  programs.  Questionnaires  are  used  for  two  purposes.  First, 
the  students  are  given  an  opportunity  to  critique  the  course  they  have  just 
completed.  Second,  operations  training  program  personnel  are  surveyed 
by  air  training  command  (ATC)  and  TAC  institutional  personnel  to  determine 
if  student  skills  and  knowledge  meet  expectations. 
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Institutional  training  program  managers  are  as  responsive  to  this  informa¬ 
tion  as  they  can  be  within  the  limits  of  policy  and  available  resources.  But 
several  factors  work  against  the  utility  and  validity  of  the  information 
produced  via  the  questionnaires.  The  students  are  not  always  in  a  position 
to  judge  whether  they  have  been  trained  effecively  or  to  a  sufficient  level 
of  proficiency.  .Also,  they  tend  to  be  less  outspoken  and  critical  for  fear 
of  adverse  effects  on  their  record.  But  these  factors  do  not  have  an 
especially  noticeable,  negative  impact. 

The  nature  of  feedback  from  the  field  is  more  severe  in  its  impact.  Each 
operational  location  has  local  operating  procedures,  some  of  which 
represent  special  cases  and  therefore  deviate  from  standard  procedures 
taught  in  the  institutional  environment.  Furthermore,  because  controlling 
aircraft  is  in  large  part  technique,  the  biases  of  individual  controllers  come 
into  play  in  any  criticism  that  is  offered.  Different  units  may  come  to 
represent  different  schools  of  thought  on  techniques  for  controlling,  depending 
on  experiences.  .AW.ACS,  S.AGE  units,  and  T.ACS  units  are  subject  to  this 
latter  problem,  but  local  operating  procedures  only  impact  S.AGE  and  T.ACS 
units.  The  institutional  environment  emphasizes  standardization  to  an 
even  greater  degree  than  the  operational  environment.  Consequently, 
institutional  training  cannot  be  responsive  to  these  demands  regarding 
procedures  and  techniques.  The  net  result  is  that  all  feedback  from  the 
field  is  taken  with  a  "grain  of  salt.  "  The  danger  is  that  institutional  training 
programs  can  wind  up  operating  in  an  open-loop  situation  and  take  on  a 
separate  existence. 
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Incomplete  Use  of  Existing  Data  in  Program  Evaluation 


The  after-action  report  (AAR)  is  not  used  to  its  fullest  advantage  in 
program  evaluation  and  modification.  The  AAR  is  used  by  stan/e vai 
personnel  in  operational  units  to  characterize  the  system  exercise  performance 
of  the  unit.  An  individual  AAR  may  have  an  impact  on  the  design  of  the 
subsequent  exercise.  In  any  event,  the  deficiencies  noted  in  the  report  are 
the  subject  of  review  and  corrective  action.  However,  despite  their 
utility,  the  AARs  serve  only  as  a  "one-shot"  mechanism.  Improved 
compilation  of  AARs  would  yield  a  data  base  of  valuable  information 
regarding  team  and  system  performance. 

2 

Low  Retention  of  Experienced  C  System  Personnel 

Low  retention  rates  are  apparently  the  result  of  such  factors  as  low  pay, 

poor  work  conditions  or  assignments,  little  job  satisfaction,  and  the  lack 

of  clear  career  paths  and  attainable  objectives.  The  importance  of 

experienced  individuals  to  successful  institutional  and  operational  training 

and  to  the  achievement  and  maintenance  of  combat  readiness  cannot  be 

overestimated.  Definition  and  development  of  training  requires  subject 

2 

matter  expertise  in  C  systems  and  tasks;  the  primary  method  of  training 

is  the  master-apprentice  model;  program  evaluation  depends  upon  the 

professional  opinion  of  qualified  experts;  and  so  on.  Clearly,  the  loss  of 

experienced  personnel  represents  a  threat  to  the  vitality  and  effectiveness 
2  2 

of  AFC  T  . 
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Shortage  of  Live  Flying  Events/ Activities  for  T 

It  is  often  claimed,  and  our  survey  was  no  exception,  that  there  is  no 
substitute  for  the  experience  of  controlling  live  aircraft.  The  same 
level  of  stress,  the  so-called  "pucker"  factor,  it  is  argued,  cannot  be 
produced  through  simulation.  An  air  weapons  controller  must  know  that 
lives  are  on  the  line  to  find  out  if  she/he  can  indeed  control  proficiently. 

The  same  notion  applies  to  teams,  also,  in  the  sense  of  survival  and  mission 
success.  Obviously,  live  flying  is  necessary  for  evaluative  purposes  as 
well. 

2 

Given  the  necessity  of  live  events  for  C  training  and  evaluation,  some 

problems  have  resulted  from  force  and  budget  reductions  in  recent  years. 

These  would  seem  to  be  constraints  that  must  be  lived  with.  Fuel  for 

aircraft  will  not  get  cheaper.  The  pr*ospect  of  increased  live  flying  to 
2  2 

support  C  T  is  unlikely. 

Both  simulated  and  live  flying  events  are  needed,  given  the  current  state 

of  knowledge.  A  great  deal  can  be  learned  from  simulated  events.  All 

2 

elements  of  the  tasks  for  the  C  operators  can  be  represented  because  their 

tactical  world  consists  of  radar  images,  graphic  plots,  and  voice/data 

links.  However,  the  levels  of  stress  and  realism  (if  any)  that  are  associated 

with  "real"  events  cannot  be  representated  unless  the  operators  cannot 

tell  the  difference  between  live  and  simulated  events  and  believe  or  act  as 

if  all  events  are  real.  Until  knowledge  is  acquired  on  the  contributions  of 

live  events  to  training  effectiveness,  an  informed  decision  cannot  be  made 

2  2 

to  eliminate  live  flying  for  C  T  . 
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Ironically,  empirical  research  to  determine  the  relative  contributions  of 
live  and  simulated  events  to  training  and  proficiency  would  require  live 
resources  in  a  quantity  that  is  probably  unattainable.  In  addition,  the  cost 
of  obtaining  simulators  with  high  enough  fidelity  to  warrant  investigation 
is  prohibitive  given  present  resources  and  priorities.  Both  are  needed.  A 
strong  data-based  advocacy  is  also  needed  to  achieve  the  appropriate 
allocation  of  resources. 


Lack  of  Instructor  Training  and  Evaluation 
in  Operations  Training  Programs 

Our  understanding  of  the  comments  of  our  respondents  is  that  instructors 
in  operations  training  programs  typically  receive  no  formal  training  in  how 
to  train,  although  they  might  have  previous  experience  as  an  instructor. 
They  are  chosen  primarily  on  the  basis  of  their  positional  proficiency  and 
are  not  evaluated  on  their  ability  as  instructors.  This  situation  places  both 
instructor  and  student  at  a  disadvantage.  The  former  may  feel  inadequately 
prepared  to  do  the  job,  and  the  latter  could  suffer  as  a  consequence. 
Improvement  in  this  area  should  be  possible  if  policy  decisions  establishing 
instructor  training  requirements  are  made. 

j  Difficulties  Posed  in  Evaluating 

!  "Soft*1  as  Opposed  to  "Hard"  Teams 

i  ’ ~  ~ 

A  "soft"  team  is  one  whose  members  change  from  mission  to  mission;  a 
1  "hard"  team  stays  together.  Little  is  known  about  the  effect  of  team 

f 

stability  on  skill  acquisition  or  team  performance. 

j 

j 


The  position  put  forth  there  is  that  stability  of  membership  is  needed  early 
in  training;  constant  switching  of  team  members  would  tend  to  retard 
skill  acquisition.  But,  in  the  long  term,  rotation  of  membership  will 
produce  more  adaptable  team  members  with  a  higher  level  of  team  skills. 

.Although  it  is  possible  to  evaluate  a  team's  performance  given  proper 
criteria  and  standards,  it  is  unclear  what  the  meaning  of  such  an  evaluation 
is  if  the  team  is  composed  of  individuals  who  work  together  on  a  one-time 
basis.  This  is  especially  disconcerting  if  one  is  attempting  to  measure  and 
generalize  about  readiness.  The  effect  of  the  soft  team  policy  on  the 
evaluation  of  readiness  should  be  determined  and  used  to  evaluate  the 
position.  If  the  policy  must  be  retained,  new  techniques  of  evaluation 
should  be  explored. 

.Answers  to  these  questions  are  imperative  to  provide  guidance  for  personnel 
policy  regarding  team  stability. 

2  2 

Lack  of  T  Guidance  for  C  Training  Program  Manager 
2  2 

AFC  T  program  managers  are  poorly  prepared  to  do  their  job.  .Among 
2 

other  things,  C  training  program  managers  are  inadequately  informed 
2  2 

about  the  C  T  training  pipeline,  about  the  different  performance  contexts 
(individual,  subteam,  team,  and  superteam;  see  Figure  3),  and  about 
the  importance  of  developing  individual  skill  proficiency  prior  to  initiating 
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2 

Despite  the  lack  of  a  generally  accepted,  comprehensive  T  philosophy, 

some  principles  and  practices  do  exist  which  it  would  be  advantageous  to 

2  9 

know  or  follow.  That  such  guidance  is  unavailable  to  C  T“  program 
managers  in  a  directly  usable  form  means  that  they  are  missing  an  opportunity 
to  be  more  effective  in  their  jobs. 

RECOMMENDATIONS  FOR  RESOLVING  ISSUES  AND  PROBLEMS 

2  2 

There  are  three  sets  of  recommendations  for  improvement  of  AFC  T  : 

1.  Near-term  solutions  attainable  through  the  immediate  application 
of  available  technology 

2.  Long-term  research  topics  to  address  issues  and  problems 
which  require  additional  research  and  development 

3.  Functional  and  operational  requirements  for  a  simulation  facility 

2  2 

for  AFC  T 

The  recommended  topics  for  near-term  and  long-term  programs  are 
presented  in  Table  6,  along  with  the  original  set  of  problems  and  issues 
from  the  survey  data. 

The  issues  and  problem  areas  identified  can  be  addressed  through  a 

research  program  aimed  at  specific  topics  in  human  factors  design, 

instructional  systems  development,  training  methodology,  *  jrformance 

measurement,  and  the  like,  or  in  some  cases  through  the  immediate 

application  of  available  technology.  Table  6  presents  recommendations 
2  2 

for  AFC  T  regarding  long-term  research  topics  and  objectives  and  near- 
term  development  needs.  The  recommendations  are  indexed  to  the  issues 


RECOMMENDATIONS  FOR  I.ONG-TERM  RESEARCH 
AND  NEAR-TERM  APPLICATION  IN  AFC2T2 


and  problem  areas  they  address.  It  is  the  intent  of  these  recommendations 

that  the  long-  and  near-term  research  topics  and  development  needs  be 

undertaken  in  parallel  as  part  of  a  larger  program  aimed  at  developing  a 
2 

model  of  C  team  performance  as  described  below. 

The  third  set  of  recommendations  concerns  the  functional  and  operational 

requirements  for  a  general-purpose  simulation  facility  that  will  be  used  to 

2  2 

explore  the  applicability  of  advanced  simulation  technology  to  .AFC  T 

research  issues.  In  addition  to  its  primary  function,  which  is  to  support 

2  9 

empirical  research,  the  simulator  will  also  serve  as  a  prototype  AFC  T” 
training  device. 
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CHAPTER  II 


RECOMMENDATIONS  FOR  THE  APPLICATION 
OF  CURRENT  TECHNOLOGY 


The  survey  data  indicated  that  11  specific  topics  could  be  addressed  through 
the  application  of  current  technology.  These  topics,  listed  in  order  of 
priority  are: 

2 

1.  C  team  supervisor  training 

2.  Supplemental  initial  training 

3.  Compilation  procedure  for  after-action  reports 

4.  Simulated  combat  mission  guidelines /aids  for 
establishing  objectives 

5.  Readiness  criteria 

2 

6.  C“  personnel  attitude  survey 

7.  Training  for  interceptor  pilot  simulators 

8.  Program  evaluation  questionnaires 

9.  Operational  unit  instructor  training 

10.  Task  analysis  method 

11.  Team  skill  taxonomy 


* 


Prioritization  of  these  topics  was  of  critical  interest  to  the  .Air  Force.  The 
dimensions  of  feasibility,  utility,  usability,  probability  of  success,  and 
practical  payoff  were  evaluated  for  each  topic  in  the  context  of  a  proposed 
Phase  II  effort.  The  criterion  of  feasibility  under  these  conditions  was 
defined  in  terms  of  the  time  and  labor  available  for  Phase  II  as  compared 
to  that  judged  necessary  for  accomplishment  of  the  application  topic.  The 
criterion  of  usability  was  defined  in  terms  of  the  appropriateness  of  the 
topic  for  inclusion  in  a  source/ reference  book  that  would  be  used  by 
training  personnel.  The  remaining  criteria  had  standard  meanings. 

The  recommended  objective,  approach,  methods,  and  resources /facilities 
for  addressing  each  problem  are  discussed  in  Volume  III  and  summarized 
in  the  present  chapter. 

C2  TEAM  SUPERVISOR  TRAINING 

A  training  program  aimed  at  improving  the  decision  making  and  team 

2 

coordination  skills  of  C  team  supervisors  could  potentially  enhance  the 

2 

operational  effectiveness  of  C  teams  because  knowledgeable,  skilled 

leaders  are  essential  for  effective  team  performance.  Unfortunately, 

2 

however,  C  team  supervisors  currently  receive  little  formal  training. 

Therefore,  the  objective  of  the  first  effort  would  be  to  demonstrate  the 

2 

feasibility  of  developing  training  material  for  a  C  team  supervisor,  the 
AWACs  Mission  Crew  Commander  (MCC),  and  evaluate  that  material 
for  appropriateness  and  training  value.  A  parallel  objective  is  to  document 
the  procedures  for  developing  and  evaluating  the  training  material.  The 
approach,  method,  and  resources/facilities  are  summarized  below: 
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•  Approach- -The  approach  would  follow  that  used  by  McCutcheon 
and  Brock  (Reference  10)  in  their  highly  successful  effort  to 
develop  training  for  the  Navy  Combat  Information  Center  Watch 
Officer  (CICWO).  The  steps  are  as  follows: 

1.  Identify  critical  job  tasks 

2.  Identify  skill  categories 

3.  Establish  skill  acquisition  guidelines 

4.  Define  exercise/experience  characteristics 

5.  Select  training  media 

6.  Develop  set  of  scripted  exercises 

7.  Produce  training  materials 

8.  Evaluate  materials 

The  proposed  effort  would  focus  on  those  job  tasks  requiring  team- 
oriented  decision-making  and  managerial  skills  in  nonstandard 
operational  situations,  or  situations  for  which  operating  procedures 
have  not  yet  been  developed. 

•  Method-  -  The  identification  of  these  types  of  job  tasks  would 
require  on-site  data  collection  and  the  assistance  of  an  operationally 
ready  MCC  who  has  a  background  in  training. 

The  data  collection  effort  would  involve  observation  of  live  and 
simulated  missions  anc  rterviews  with  operationally  ready  MCCs. 
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The  specification  of  training  guidelines  and  media  would  be 
reviewed  with  Air  Force  subject  matter  experts. 

The  evaluation  of  the  training  materials  produced  would  require 
on-site  data  collection  consisting  of  demonstration  and  use  of 
the  materials.  Both  student  and  operationally  ready  MCCs 
would  participate  in  the  evaluation.  The  primary  evaluative 
tools  would  be  observation  of  use  and  administration  of  a 
questionnaire. 

•  Required  Resources /Facilities- -Three  site  visits  to  Tinker  AFB 
would  be  required.  A  subject  matter  expert  would  be  required 
for  occasional  telephone  consultation.  The  production  of  audio 
and/or  visual  materials  from  approved  contractor  written  scripts 
would  be  accomplished  by  Air  Force  audio/visual  experts. 

SUPPLEMENTAL  INITIAL  TRAINING 

Successful  AWCs  must  be  skillful  in  mathematics,  spatial  reasoning,  and 
communication  in  English.  There  are  no  career  field,  entry  level  minimum 
requirements  for  these  skills.  At  the  present  time  the  Air  Force  cannot 
be  selective  as  a  result  of  manpower  shortages.  The  objective  of  the  second 
research  effort  would  be  to  supplement  initial  AWC  training  with  relevant 
remedial  instructional  programs.  The  potential  payoff  would  be  to  reduce 
the  washback  and  washout  rates  for  weapon  controllers  and  increase  their 
operational  effectiveness. 
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•  .Approach- -The  first  step  would  be  to  assess  deviciencies  in 
mathematical  and  communication  skills.  Once  deficiencies  are 
pinpointed,  the  appropriate  supplemental  training  needs  could  be 
identified.  It  is  likely  that  programmed  instruction,  self-study 
modules  already  exist  for  some  skills  required;  and  they  could 
be  implemented  immediately. 

•  Method- -Existing  test  records--for  example,  .Air  Force  officer 
qualifying  test  (.AFOQT)  scores--would  provide  data  on  skill  levels 
of  past  and  present  students.  Trends  could  be  observed. 
.Alternatively,  specific  achievement  tests  could  be  given  to  samples 
of  the  student  population,  recent  course  graduates,  operationally 
ready  controllers,  and  standardization/ evaluation  controllers. 
Differences  among  the  groups  would  allow  deficiencies  to  be 
pinpointed. 

•  Required  Resources /  Fa cilities- -Computer  support  would  be 
required  for  data  retrieval  and  analysis.  .Access  to  personnel 
records  would  be  required.  Standardized  tests  might  need  to  be 
procured  and  administered.  Training  material  would  have  to  be 
developed  if  existing  material  was  inadequate. 

COMPILATION  PROCEDURE  FOR  .AFTER- .ACTION  REPORTS 

2 

The  .A.AR  is  not  used  to  fullest  advantage  in  T  program  evaluation,  primarily 
because  it  serves  only  as  a  "one-shot"  mechanism.  The  objective  of  the 
third  research  effort  would  be  to  develop  a  procedure  for  compiling, 
analyzing,  and  synthesizing  a  series  of  AARs.  The  data  base  so  produced 
could  be  used  in  numerous  ways  including  the  following: 
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1.  Improve  the  AAR  format  so  that  it  more  directly  addresses 
feedback  to  teams  about  team-oriented  skills. 

2.  Identify  issues  requiring  improved  training  or  modified  or 
new  operating  procedures. 

3.  Identify  issues  of  system  design  which  affect  team  performance. 

4.  Provide  a  reference  for  new  team  members,  especially 
supervisors. 

5.  Provide  a  rich  source  for  specific  research  issues. 

The  approach,  method,  and  resources/facilities  are  summarized  below: 

•  Approach- -The  approach  would  be  to  employ  an  analytic  technique 
for  sorting  the  AAR  data  into  meaningful  categories.  The  data 
would  then  be  evaluated  in  order  to  pinpoint  operational  deficiencies 
and  their  causes.  The  consequences  of  the  deficiencies  would  be 
addressed  in  terms  of  training  requirements  or  corrective  action. 

•  Method — The  AAR  data  would  be  sorted  into  task/skill  categories 
within  different  performance  contexts.  Figure  5  shows  the  three 
performance  contexts  (preteam,  team,  and  superteam)  and  the 
task/ skill  categories  (mission  and  team)  which  would  be  used. 


The  mission  tasks  are  operationally  defined  by  the  system  and 
the  particular  scenario.  The  team-oriented  tasks  include  the 
following: 


PERFORMANCE  CONTEXT 

PRE-TEAM  TEAM  SUPERTEAM 


MISSION 

TASK/ 

SKILL 

CATEGORY 

TEAM- 

ORIENTEO 


Figure  5.  Analysis  Framework  for  After-Action  Reports 

--Communications  (discipline) 

--Adaptability  to  emergent  events 

--Anticipation  (consequences  of  own  action  or  information 
needs  of  others) 

--Adjustment  to  workload  modifications 
—  Problem  solving/ decision  making 

Figure  6  shows  the  framework  that  would  be  used  to  synthesize 
the  compiled  data.  There  are  three  different  elements  which 
may  be  characterized  in  terms  of  strengths,  deficiencies,  and  the 
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ELEMENT 


STRENGTHS 


DEFICIENCIES 


OPERATIONAL 

CONSEQUENCES 


OPERATING 

HUMAN  SYSTEM  PROCEDURES 


TRAINING  NEEDS 
CORRECTIVE  ACTIONS 


Figure  6.  Synthesis  Framework  for  Compiled  AAR  Data 

operational  consequences.  These  elements  are  human,  system 
(hardware  and  software),  and  operating  procedures  which  vary 
from  well  defined  (established)  to  ill-  or  non-defined  (emergent). 
The  operational  consequences  of  deficiencies  will  lead  to  rec¬ 
ommendations  regarding  training  needs  or  corrective  actions 
involving  perhaps  system  design  (future  systems,  for  example) 
or  the  development  or  modification  of  operating  procedures. 

•  Required  Resources /Facilities --Development  of  the  compilation 
procedure  would  require  the  cooperation  of  a  standardization/ 
evaluation  section  of  the  target  system  (a  CRC  would  be  ideal). 
Access  to  AAR  data  would  be  required,  and  review  with  Air 
Force  subject  matter  experts  would  be  needed. 
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GUIDELINES /AIDS  FOR  DEFINING  SIMULATED 
COMBAT  MISSION  OBJECTIVES 


Current  methods  for  defining  objectives  and  characteristics  of  simulated 
combat  missions  are  unsystematic.  The  complexities  of  the  planning  task 
might  exceed  man's  unaided  information  management  abilities.  The 
objective  of  the  fourth  research  effort  would  be  to  develop  a  set  of 
procedural  guidelines,  which  ultimately  might  lead  to  automation,  for  the 
definition  of  features  and  training  objectives  for  simulated  combat  missions. 

•  Approach- -The  first  step  would  be  to  understand  how  planning  is 
currently  done  ana  the  constraints  placed  on  the  definition  of 

the  objectives  and  characteristics  of  simulated  combat  missions. 
The  intended  users  of  the  guidelines  and  aids  would  participate  in 
their  development  to  as  great  an  extent  as  possible. 

•  Methods- -Observation  of  current  planning  conferences  would  be 
essential.  The  structure  of  existing  procedures  would  be  captured 
as  completely  as  possible.  The  guidelines  and  aids  would  be 
presented  in  a  procedural  format  within  a  source /reference  book 
for  exercise  planners.  The  guidelines  would  be  assessed 
through  demonstration  and  use  in  a  representative  planning 
conference. 

•  Required  Resources /Facilities- -Access  to  at  least  two  planning 
conferences  and  at  least  one  demon  ‘■■"ation/evaluation  would  be 
required. 


READINESS  CRITERIA  DEVELOPMENT 


Current  evaluative  techniques  for  assessing  team  readiness  are  either 
too  objective,  with  little  information  value,  or  too  subjective  to  be  of  use 
in  training  effectiveness  assessment.  The  objectives  of  the  fifth  research 
effort  would  be  to  identify  the  dimensions  of  team  performance  to  which  an 
expert  evaluator  attends,  and  to  articulate  the  criteria  the  expert  applies  in 
judging  team  readiness.  The  payoff  would  be  the  development  of  methodological 
tools  for  assessing  team  readiness  and  training  effectiveness. 

•  Approach- -The  approach  would  make  use  of  the  techniques  of 
multi -attribute  utility  theory  and  apply  them  to  teach  evaluation 
decision  making.  Expert  evaluators  would  serve  as  the  source 
for  extracting  independent  dimensions  which  they  consider 
important  and  obtaining  assessments  of  the  value  or  utility  of 
particular  attributes  on  the  dimensions. 

•  Method- -Extensive  observation  of  and  interviews  with  expert 
evaluators  would  be  required.  Automation  of  the  decision 
situation  structure  would  be  necessary.  A  representative  and 
general  solution  could  be  achieved  by  using  an  element  of  the 
TACS  as  the  object  of  the  development. 

•  Required  Resources /Facilities --Cooperation  and  participation 
of  standardization/ evaluation  personnel  would  be  required.  The 
participation  of  an  operational  unit  (for  example,  CRC)  would  be 
necessary.  A  mainframe  computer  and  appropriate  software 
would  be  required  for  decision  model  development. 


CONDUCT  A  C2  PERSONNEL  ATTITUDE  SURVEY 

2 

Retention  of  experienced  individuals  and  morale  in  the  C  career  field  are 
low.  The  objective  of  the  sixth  research  effort  is  to  obtain  useful  data  for 
a  review  of  personnel  policies.  The  payoff  would  be  recommendations  for 
improving  personnel  policies. 

•  Approach- -The  survey  would  be  conducted  utilizing  state-of- 

the-art  employee  attitude  survey  techniques.  A  concerted 

survey  effort  would  provide  a  necessary  data  base  for  policy 

2 

decisions,  plus  indicate  to  the  C  community  that  the  "employer'1 
cares. 

•  Method-  -  The  survey  instrument  would  have  to  be  designed.  The 
design  could  be  based  on  existing  instruments  from  industry 
sources.  Data  reduction  would  require  automated  data  processing 
support.  Recommendations  would  be  based  on  collected  data. 

•  Required  Resources/Facilities-- A  mainframe  computer  and 
appropriate  software  would  be  required. 

TRAINING  FOR  INTERCEPT  PILOT  SIMULATORS 

Because  personnel  who  simulate  interceptor  pilots  receive  no  formal 

training,  the  training  experience  for  AWCs  is  suboptimai.  The  objective 

of  the  seventh  research  effort  would  be  to  develop  formal  training  materials 

9  2 

for  IPS  personnel  in  order  to  improve  the  quality  of  current  AFC“T 
programs. 
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•  Approach--The  tasks  of  the  IPS  would  have  to  be  specified  and 
training  requirements  established.  Training  materials  should 
be  of  the  self-study  variety. 

•  Methods- -ISP  methodology  should  provide  the  proper  methods  for 
this  application. 

•  Required  Resources  /Facilities- -Access  to  "good"  IPS  personnel 
would  be  required. 

IMPROVED  PROGRAM  EVALUATION  QUESTIONNAIRES 

Current  questionnaire  approaches  to  initial  and  initial  transition  training 
program  evaluation  are  of  suspect  validity.  The  objective  of  the  eighth 
research  effort  would  be  to  produce  a  questionnaire  which  would  increase 
the  quality  and  quantity  of  the  evaluative  data  the  schools  currently  receive 
from  the  field.  This  would  in  turn  potentially  improve  the  quality  of 
school  programs. 

•  Approach- -A  better  articulated  purpose  and  structure  of  the 
questionnaire  technique  would  enable  field  personnel  to  address 
issues  related  to  team-oriented  skills.  More  quantitative  data 
regarding  student  success  should  be  sought. 

•  Method — A  review  of  current  questionnaires  should  be  undertaken. 
Techniques  of  the  growing  area  of  program  evaluation  would  be 
applied  to  the  modification/ enlargement  of  the  questionnaire. 

The  questionnaire  would  be  utilized  to  determine  if  it  provides  new 
or  usefully  formatted  evaluative  information. 
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•  Required  Resources/Facilities--Cooperation  and  participation  of 
institutional  and  operational  personnel  would  be  required  in 
producing  the  questionnaire  and  conducting  the  survey. 

DEVELOP  OPERATIONAL  UNIT  INSTRUCTOR  TRAINING  PACKAGE 

Instructors  in  operations  training  programs  receive  no  formal  training  in 
how  to  train.  The  objective  of  the  ninth  research  effort  would  be  to  develop 
a  training  package  for  instructors  in  operational  units.  The  payoff  would 
potentially  be  improved  training  effectiveness  and,  ultimately,  improved 
team/ system  effectiveness. 

•  Approach- -The  ATC  four-week  instructor  training  course  could 
be  used  for  materials.  A  mobile  training  team  could  teach  a 
condensed  course,  or  a  self-study  package  could  be  offered  as 

a  correspondence  course. 

•  Method- -Standard  instructional  material  development  techniques 
would  be  applied  making  as  much  use  of  existing  materials  as 
possible. 

•  Required  Resources/ Facilities- -Participation  of  ATC  training 
developers  would  be  desirable. 

TEAM  TASK  ANALYSIS  METHODOLOGY 

Current  task  analytic  techniques  provide  for  analysis  of  operators' 
individual  tasks  and  fail  to  identify  team-oriented  skills.  The  objective  of 
the  tenth  research  effort  would  be  to  develop  a  task  analytic  methodology 
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for  producing  detailed  descriptions  of  tasks  requiring  interpersonal 

dependencies  and  interactions.  The  methodology  would  be  oriented  toward 

2 

improving  the  team  processes  of  AFC  teams. 

•  Approach — Available  task  analytic  formats  (for  example,  ISD  or 
Data  Item-H-6130)  would  be  expanded.  Guidelines  would  be 
developed  permitting  the  identification  of  team-oriented  tasks. 

•  Method- -Determination  of  conditions  and  standards  of  performance 

would  require  subject  matter  expertise  more  related  to  efficient 
2 

and  effective  C  team  performance  than  to  the  specific  system  or 
mission  tasks.  The  experts  should  be  generalists  of  supervisory 
rank  rather  than  technical  specialists.  The  specific  details  of 
this  effort  would  depend  on  successfully  developing  a  team  task 
taxonomy  as  is  described  next. 

•  Required  Resources/Facilities--Participation  of  subject  matter 
experts  would  be  required. 

TEAM  BEHAVIOR  TAXONOMY  DEVELOPMENT 

Current  ISD  guidelines  and  procedures  do  not  address  team-oriented  tasks 
and  skills.  The  objective  of  this  effort  would  be  to  develop  a  behavioral 
taxonomy  which  forces  attention  to  team-oriented  tasks  in  the  initial 
steps  of  the  ISD  procedure.  The  payoff  for  this  effort  would  be  a  procedural 
framework  ensuring  the  consideration  of  team  performance  in  institutional 
system  development. 


Approach — The  approach  would  make  use  of  existing  behavior 
taxonomies  to  as  great  an  extent  as  possible.  A  revised  taxonomy 
would  include  team  behavior  categories  like  anticipate,  coordinate, 
initiate/ receive  (information),  etc. 

2 

Method- -Observation  of  actual  C  teams  carrying  out  simulated 
combat  missions  would  be  undertaken  to  develop  and  validate  the 
taxonomy.  To  achieve  representativeness  and  generality,  the 
TACS  CRC  should  provide  the  observation  environment.  Data 
descriptive  of  task  frequency  over  mission  segments  would  be 
co  lie  cted. 

Facilities- -Cooperation  of  an  operational  unit  would  be  required. 

A  hand-held  data  recorder  and  a  mainframe  computer  for  data 
reduction  would  be  necessary. 


52 


CHAPTER  IH 


RECOMMENDATIONS  FOR  AN  AFC2T2  RESEARCH  PROGRAM 

Recommendations  for  long-term  research  projects  were  derived  by  a 
three-step  process.  Long-term  research  topics  were  formulated  to  address 
the  issues  and  problem  areas.  These  topics  were  then  analyzed  to  generate 
research  projects.  Sixteen  projects  were  recommended  as  candidates. 

Table  7  contains  a  listing  of  the  issues  and  problems  from  Volume  I  and 
the  long  term  research  topics  and  candidate  research  projects  from 
Volume  II. 

The  third  step  was  an  evaluation  of  candidate  research  projects  for 
implementation.  These  were  rated  on  the  criteria  of  technical  feasibility, 
utility,  usability,  probability  of  success,  and  practical  payoff  and  were 
ordered  in  priority  on  the  basis  of  the  ratings.  The  evaluation  process  and 
results  are  presented  in  Volume  II,  Chapter  III. 

Each  of  the  candidate  research  projects  is  described  briefly  in  the  following 
pages.  They  are  taken  in  the  order  of  the  listing  in  Table  7. 

FIELD  STUDY  FOR  PRECISE  DEFINITION  OF  DEFICIENCIES 
IN  C2  TEAM  PERFORMANCE  AND  TRAINING 

The  objective  is  to  obtain  more  precise  definition  of  the  deficiencies  found 
during  this  study  by  using  more  objective  and  more  focused  evaluation  of 
each  issue.  Better  estimation  of  the  impact  on  effectiveness  and  cost- benefit 
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TABLE  7.  RECOMMENDATIONS  FOR  LONG-TERM  RESEARCH 


data  could  be  obtained.  The  survey  in  the  present  study  provided  information, 
but  the  major  accomplishment  was  to  identify  problems  and  needs.  The 
information  obtained  was  not  sufficiently  detailed  to  permit  analysis  of 
the  problems  in  depth.  A  subsequent  study  could  build  on  the  survey, 
using  the  findings  as  hypotheses,  and  develop  probes  to  provide  deeper 
analysis. 

Data  collection  would  have  to  take  place  in  operational  units  and  schools 
during  preteam,  team,  and  superteam  exercises.  Performance  objectives 
will  have  to  be  developed  because  existing  objectives  and  standards  are 
unlikely  to  be  adequate  in  terms  of  coverage,  comprehensiveness,  metric 
properties,  interpretability,  and  appropriateness  for  the  research  purposes. 

The  potential  impact  of  these  deficiencies  on  system  effectiveness  can  be 
assessed,  and  cost  estimates  could  be  made  for  specific  research  studies 
and  projects.  They  can  then  be  evaluated  on  the  basis  of  cost-benefit 
comparisons.  The  results  of  the  study  would  feed  into  all  other  research 
activities. 

CONSTRUCT  A  MODEL  FOR  C2  TEAM  PERFORMANCE  IN 
CRC,  AWACS,  OR  TACC  TYPE  OF  ORGANIZATION 

2 

The  objective  is  to  take  the  first  step  in  developing  C  team  performance 

models.  The  purpose  of  the  model  is  to  provide  a  conceptual  framework 

2 

and  terminology  for  describing  and  analyzing  the  behaviors  of  C  teams. 

The  approach  consists  of  observation  of  team  operations  and  analysis  of 

2 

the  technical  literature  on  teams  and  C  to  develop  and  test  a  limited  model 
based  on  a  selected  system. 


Development  of  a  framework  for  team  phenomena  is  a  fundamental  need 
that  underlies  ail  research  topics.  The  lack  of  a  systematic,  comprehensive 
framework  for  describing  and  analyzing  teams  and  human  factors  require¬ 
ments  for  team  operation  is  a  fundamental  deficiency  underlying  all  research 
issues  on  teams.  This  deficiency  is  manifested,  for  example,  in  a  lack  of 
adequate  terminology  and  taxonomies  for  describing  teams,  behavioral 
phenomena  in  teams,  types  of  activities  done  by  teams,  types  of  teams 
that  occur  in  military  systems,  and  differentiation  between  teams  and 
multi- individual  aggregates  of  people  working  independently.  The  model 

will  provide  a  common  reference  for  research,  technology  development, 

2  2 

and  design  for  C  T  . 

2 

The  objectives  of  developing  a  C  team  performance  model  would  be  to 
establish  the  nature  of  teams  in  four  ways: 

1.  Structure  of  teams  in  terms  of  a  set  of  dimensions  or  attributes 
on  which  teams  can  vary  with  a  corresponding  variation  in  team 
performance. 

2.  Processes  by  which  teams  accomplish  their  functions  and  work. 

3.  Internal  and  external  variables  that  affect  the  processes  and 
performance  of  teams. 

4.  Team  behaviors  and  skills  in  terms  of  which  activities  of  teams 
car  be  described. 

2 

A  preliminary  C  team  model  was  presented  consisting  of  four  principal 
modules:  team,  tactical  situation,  command  team  behavior,  and  driver 
scenario.  The  team  module  was  described  in  terms  of  architectural/structural 
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dimensions  and  functional  properties.  The  former  concern  team  structure, 
mission  tasks,  and  operating  procedures.  The  latter  concern  the  process 
by  which  the  team  performs.  Operational  tasks  are  related  to  the 
architectural/ structural  dimensions  while  team  behaviors  are  related  to 
the  function's  properties.  Team  behaviors  are  the  factors  that  differentiate 
teams  from  non-teams,  and  they  are  a  function  of  a  shared  plan  or  scheme. 

The  tactical  situation  module  represents  the  tactical  plan,  its  objectives, 
and  the  changing  state  of  the  battlefield  as  the  plan  is  implemented.  The 
command  team  behavioral  module  models  the  functions  and  tasks  performed 
by  members  of  the  command  team  (battle  staff).  The  driver  scenario 
module  generates  the  enemy  and  other  events  which  change  the  state  of  the 
world. 

Methodological  developments  in  behavioral  analysis  and  performance 

measurement  would  be  made  possible  through  and  during  model  development. 

2 

These,  in  turn,  would  be  applied  to  the  design  and  support  of  C  systems. 

DEVELOP  PROCEDURES  FOR  SYSTEM  AND  TASK  ANALYSIS, 
COMPATIBLE  WITH  MIL-H-46855  and  ISD,  TO  BE  USED  IN  DESIGN 
OF  MAN-MACHINE  COMPONENTS  FOR  C2  TEAMS 

The  objective  is  to  develop  tools  for  generating  and  analyzing  behavioral 
data  to  design  man-machine  interfaces,  personnel  subsystems,  and  training 
during  system  development  and  maintenance  of  system  readiness.  These 
tools  will  be  adaptations  or  extensions  of  existing  methods  to  the  less 
structured  situation  of  command  and  control  systems  and  team  performance. 
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The  approach  consists  of  identifying  the  sequence  of  analytical  issues  and 

steps  necessary  to  go  from  an  operational  need  or  system  concept  to 

2 

completion  of  development  of  the  man- machine  components  for  C  teams, 
and  developing  or  compiling  data  bases,  algorithms,  models,  aids,  and 
procedures  for  implementing  the  procedures. 

Generation  of  only  task-analytic  data  is  not  enough.  The  user  must  also 
be  provided  with  techniques  and  algorithms  for  processing  the  task  data 
to  provide  solutions  to  the  problems  of  design  of  team  structures,  man- 
machine  interfaces,  personnel  requirements,  and  training. 

This  type  of  approach  is  the  best  strategy  for  communicating  the  results 
of  the  research  to  the  community  of  designers  and  developers  of  systems. 
Providing  techniques,  data,  and  concepts  is  not  sufficient  to  have  an 
effect  on  system  development  except  in  the  very  long  term.  System 
development  is  organized  principally  by  formal  and  informal  procedures 
which  have  become  accepted  practices.  Changes  in  system  design  must 
be  achieved  by  incorporating  new  methods,  concepts,  or  data  into  these 
practices  or  replacing  part  or  all  of  an  existing  procedure. 

MIL-H-46885,  related  Data  Item  Descriptions  (DIDs),  and  human  engineering 

specifications  (1472B,  for  example)  have  acceptance  as  guidelines  for 

human  factors  area  of  design.  They  have  two  significant  shortcomings  for 
2  2 

C  T  :  1 )  They  do  not  make  explicit  provision  for  the  design  of  teams, 

and  2)  They  are  oriented  toward  the  knobs  and  dials  approach  without 

adequate  consideration  of  cognitive  performance  in  emergent  situations 

2  2  2  2 

which  characterize  C  T  .  Design  for  C  T  will  be  inadequate  until 
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requirements,  methods,  and  data  are  incorporated  into  the  standard 
practices  for  design,  development,  and  evaluation  of  systems. 

Transferring  technology  to  the  community  of  system  developers  has  been 
a  slow  and  difficult  practice.  There  appear  to  be  no  effective  channels 
for  getting  information  into  it  from  outside  sources.  There  is  a  straight¬ 
forward  flow  from  the  developmental  to  the  operational  communities. 
However,  feedback  from  the  operational  community  or  information  from 
the  technical  laboratories  and  other  external  agencies  flows  very  poorly. 

A  possible  contributing  factor  is  that  engineering  development  programs 
usually  operate  on  such  "tight"  schedules  and  budgets  that  changes  and 
innovation  in  the  process  are  discouraged.  Conventional  practices  are 
used  as  tried  and  proven  methods  with  low  risk.  Any  change  decreases 
the  probability  of  not  meeting  schedule  and  cost.  The  history  of  system 
development  shows  that  delays  and  overruns  overturn  the  program  schedule, 
but  the  practices  persist. 

Another  possible  factor  is  the  turnover  in  program  management  personnel. 
There  are  several  sets  of  managers,  planners,  and  designers  during  the 
period  from  concept  development  through  engineering  development.  If 
there  is  feedback  from  the  field,  it  will  go  to  people  other  than  those  who 
made  the  faulty  decisions.  It  exists  as  a  new  problem  instead  of  feedback 
to  the  latest  set  of  program  managers. 
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One  approach  to  this  problem  of  influencing  the  designer  process  in  the 
area  of  manpower  and  training  is  under  development.  It  is  referred  to  as 
"front-end  analysis.  "  It  is  a  form  of  human  factors /systems  analysis  of 
user  requirements.  It  has  two  objectives:  1 )  consideration  of  human 
factors  requirements  starting  with  the  concept  development  stage  at  the 
beginning  of  system  development,  and  2)  iteration  and  refinement  of  these 
requirements  at  the  other  stages.  Requirements  and  methods  are  under 
development  by  the  Department  of  Defense  (Reference  11),  Air  Force 
(Reference  12),  Army  (Reference  13),  and  Navy  (Reference  14). 

DEVELOP  A  PERFORMANCE  MEASUREMENT  SYSTEM  FOR  C2 
TEAMS  WHICH  ADDRESSES  TEAM  COMPETENCE,  EFFECT  ON 
SYSTEM  EFFECTIVENESS,  AND  NEEDS  FOR  REMEDIAL  TRAINING 

The  objective  is  to  provide  more  objective  and  systematic  measures  of 
2 

performance  of  C  tea  rs.  These  measures  are  intended  to  provide  an 
index  of  team  competence,  interpretable  in  terms  of  impact  on  system 
effectiveness,  and  diagnostic  of  team  and  individual  training  needs. 

The  availability  of  these  measures  will  provide  appropriate  system  managers 
and  planners  with  objective  data  on  unit  and  individual  capability  which  can 
in  turn  be  used  in  planning  for  operations,  training,  and  support.  Programs 
for  the  maintenance  and  improvement  in  proficiency  can  be  based  on  more 
accurate,  detailed  data  than  is  available  today. 


COMPARE  "SOFT"  VS  "HARD"  C2  TEAMS  IN 
PERFORMANCE  OF  REPRESENTATIVE  C2  TASKS 

Soft  teams  do  not  have  a  stable  membership;  team  members  change  between 

exercises.  The  objective  of  this  project  is  to  assess  the  impact  of  the 

instability  on  team  performance.  The  rotation  of  team  members  would 

retard  the  learning  of  team  skills  because  some  relevant  cues  and  responses 

would  change  between  training  sessions  as  the  team  members  change. 

Therefore,  it  is  postulated  that  stable  teams  are  needed  during  the  early 
2 

stages  of  T  .  However,  rotation  of  team  membership  should  yield  more 
stable  adaptability  team  skills  for  the  more  experienced  person.  The  skills 
become  independent  of  the  situation- specific  cues  associated  with  characteristics 
of  other  team  members. 

Verification  of  the  hypothesis  will  provide  information  for  management  and 

deployment  of  team  resources  that  will  increase  operational  effectiveness 

2  2 
of  C  teams  and  provide  a  strategy  for  T  that  will  provide  higher  levels 

of  competence. 

DEVELOP  STANDARDIZED,  REPRESENTATIVE  TASKS  FOR 
C2  TEAMS  TO  BE  USED  IN  EXPERIMENTAL  STUDIES 

2 

The  objective  is  to  develop  standardized  tasks  for  representative  C  team 

functions  to  be  used  in  experimental  studies  of  team  processes.  These 

tasks  would  be  used  subsequently  in  research  studies.  The  software  for 

them  would  become  a  library  of  task  modules  which  can  be  called  up  when 

compiling  scenarios  and  experimental  designs  for  specific  research  studies. 

2  2 

They  will  also  feed  information  on  C  tasks  into  other  projects  of  T 
research  and  application. 


DEVELOP  A  PROBLEM  SPACE  OF  CALIBRATED  EXERCISES  FOR  C2T2 


This  project  has  two  objectives:  1)  provide  link  between  training  content 
and  job  tasks  by  means  of  a  sequence  of  training  objectives  that  progressively 
approach  job  tasks,  and  2)  provide  sequences  of  training  exercises  increasing 
in  complexity  and  difficulty  and  progressing  through  higher  levels  of  skill 
and  knowledge. 

The  assumptions  underlying  current  training  technology  are  that  the  learning 
of  complex  behaviors  starts  with  learning  elements  of  knowledge  and  skill 
which  are  then  integrated  into  more  and  more  complex  behaviors.  Learning 
these  complex  behaviors  proceeds  by  taking  the  individual  through  a 
sequence  of  simulated  performance  tasks  which  progressively  approximate 
job  tasks  more  closely. 

Development  of  the  problem  space  would  impose  a  structure  on  the  problems 
in  terms  of  the  knowledge  and  skill  requirements  and  establish  a  sequential 
relationship  among  them.  It  would  increase  the  effectiveness  and  efficiency 
of  training.  Higher  levels  of  proficiency  could  be  obtained  at  less  cost. 

The  results  would  also  contribute  to  the  development  of  objective,  diagnostic 
performance  measures. 

DEVELOP  DECISION  AIDS  FOR  C2  TEAMS 

The  objective  is  to  provide  improved,  interactive  decision  support  systems 
in  man-machine  interfaces  to  facilitate  team  performance.  This  project 
is  aimed  at  effective  utilization  of  the  increasing  levels  of  automation  in 

2 

system  design.  Command  and  control  systems  typically  are  inundating  C 
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personnel  with  quantities  of  data  which  they  often  cannot  correlate  and 

2 

reduce  into  useful  tactical  information.  One  observer  commented  that  C 
systems  give  data  that  the  users  do  not  use  and  fail  to  help  them  determine 
what  they  do  need  (Reference  8). 

Increasing  levels  of  automation  are  attenuating  the  usefulness  of  the 
dichotomy  between  man  and  machine.  The  complex,  difficult  performance 
problems  in  systems  have  become  tasks  of  either  computer-aided  man  or 
man-aided  computer.  Thus,  the  design  of  man- computer  transactions 
and  provision  of  adequate  decision  aiding  are  important  issues.  The  TACC 
in  particular  is  a  system  that  will  undergo  considerable  development  in 
the  automation  and  semi-automation  of  information  processing  to  assist 
the  operators. 

2 

The  development  of  aids  will  increase  the  operational  effectiveness  of  C 

2 

systems  and  might  also  reduce  the  amount  of  T  and  numbers  of  people 
required. 

ASSESS  APPLICABILITY  OF  EXISTING  TAXONOMIES  FOR  LEARNING 
TYPES  AND  INSTRUCTIONAL  STRATEGIES  TO  DESIGN  OF  c2t2 

The  objective  is  to  more  fully  assess  the  applicability  of  current  training 

2  2 

technologies  to  C  T  .  Existing  knowledge  on  the  effectiveness  of  instructional 

methods  for  training  specific  types  of  knowledge  and  skill  have  been 

developed  for  individual  training.  These  same  methods  are  undoubtedly 
2  2 

applicable  to  C  T  ,  but  the  form  and  extent  of  their  applicability  is  unknown 

2  2 

since  the  knowledge,  skills,  and  performance  objectives  for  C  T  have  not 
been  determined. 
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This  project  would  result  in  establishing  the  relationship  between  instructional 
2 

methods  and  T  requirements  and  make  existing  technology  more  available 
2 

for  T  .  More  effective  and  efficient  training  would  then  be  possible. 

ASSESS  USEFULNESS  OF  AVAILABLE  TECHNIQUES  FOR 
SEQUENCING  INSTRUCTIONAL  CONTENT  FOR  C2T2 

The  overall  objective  of  these  projects  is  to  develop  an  instructional 
2  2 

methodology  for  C  T  .  Existing  methods  were  developed  in  the  context  of 

2  2  9 

individual  training  and  may  be  applicable  to  C  T  if  the  job  tasks  in  C~ 

teams  can  be  analyzed  to  the  necessary  level.  Assessment  of  the  applicability 

2  9 

of  these  methods  to  T  would  make  this  technology  available  for  T"  as 

o 

well  as  individual  training.  More  effective  and  efficient  T  could  then  be 
provided. 

COMPARE  C2  JOB  CONTEXT  VS  NEUTRAL  CONTEXT 
'OR  TRAINING  TEAM  SKILLS 

The  objective  is  to  compare  two  approaches  to  training  team  skills.  One 
approach  consists  of  training  them  in  the  context  of  the  job  tasks  where  they 
occur;  the  other  approach  consists  of  training  them  in  a  context  free  of 
job  context  such  as  multi-person  research  games,  so  that  the  team  skills 
can  be  more  clearly  high-lighted.  The  approaches  may  differ  in  ease  of 
learning  and  amount  of  transfer  of  training.  Context- neutral  approaches 
would  not  require  simulation  of  operational  situations.  They  might  then 
be  less  costly,  applicable  to  a  larger  population,  and  more  flexible  to  use. 
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If  there  is  a  differential  in  either  cost  or  effectiveness,  improved  training 
can  be  delivered  at  lesser  cost. 

COMPARE  COST-EFFECTIVENESS  OF  LIVE  AND 

simulated  events  for  c2t2  exercises 

The  objective  is  to  develop  tradeoff  data  for  cost-effectiveness  evaluation 

2  2 

of  simulated  events  in  C  T  exercises.  The  evaluation  can  be  used  to 
develop  a  strategy  for  using  simulation  to  reduce  costs  and  increase  the 
training  effectiveness  of  exercises. 

Reduction  in  live  flying  would  reduce  energy  costs  and  conserve  operational 
equipment.  Therefore,  it  is  important  to  determine  the  comparative 
training  value  of  live  vs  simulated  events  for  the  purpose  of  identifying 
mixes  of  them  that  maximize  training  benefit  and  reduce  costs. 

DEVELOP  SIMULATION  AND  TRAINING 
REQUIREMENTS  FOR  ECM  IN  C2T2 

The  objective  is  to  determine  the  training  necessary  for  job  performance 

in  an  ECM  environment.  This  area  is  a  critical  one  for  operational 

effectiveness,  but  one  where  performance  and  training  are  inadequate. 

2 

Solving  this  problem  area  of  T  will  produce  significant  improvements 
in  operational  effectiveness. 
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DEVELOP  SIMULATION  AND  TRAINING  REQUIREMENTS 
FOR  SENSOR  MANAGEMENT  IN  C2T2  EXERCISES 


Adding  this  capability  to  training  exercises  will  improve  the  realism, 

2  2 

fidelity,  and  effectiveness  of  C  T  .  The  area  is  a  significant  deficiency 
in  current  operational  training.  It  is  also  a  critical  area  for  operational 
effectiveness  since  it  affects  the  quality  and  utility  of  the  information 
obtained  through  a  sensor  system. 

DEVELOP  TYPE  A  SYSTEM  SPECIFICATION 
FOR  A  C2T2  SIMULATION  FACILITY 

The  purpose  of  this  project  is  to  develop  the  functional  requirements  and 

o  2 

system  concept  for  a  simulation  facility  for  training  and  research  in  C  T  . 
The  requirements  are  derived  from  an  analysis  of  the  intended  application 
and  the  capabilities  needed  to  operate  and  support  the  facility. 

This  specification  becomes  the  design  goal  for  which  more  detailed 
engineering  specifications  are  prepared  from  which  procurement  decisions 
are  made. 

The  responsiveness  of  the  simulation  facility  to  the  purposes  for  which 
it  was  procured  or  built  is  a  direct  function  of  the  comprehensiveness 
and  thoroughness  of  the  specification. 


DETERMINE  FEASIBILITY  OF  DEVELOPING  SELECTION 
CRITERIA  FOR  WEAPONS  DIRECTORS 


There  are  no  selection  criteria  for  weapons  directors  at  the  present  time 
beyond  those  for  officer  selection.  The  lack  of  appropriate  aptitudes  may 
be  a  source  of  inadequate  performance  in  weapons  directors.  The  objective 
of  this  effort  is  to  evaluate  the  feasibility  of  identifying  and  setting  entry 
standards  for  the  career  field  and  thereby  improving  operational  performance. 
Existing  standards  of  performance  and  assessment  techniques  will  probably 
not  be  adequate  for  this  research,  and  a  suitable  one  will  have  to  be 
developed. 

If  some  selection  or  establishment  of  minimal  entry  requirements  on 
basic  skills  can  be  set,  then  a  higher  level  of  proficiency  can  be  obtained, 
and  training  time  and  costs  can  be  reduced. 

RECOMMENDED  RESEARCH  TOPICS 

The  candidate  research  topics  were  evaluated  against  five  criteria  by 
two  human  factors  specialists.  The  criteria  were  feasibility  of  accomplish¬ 
ment,  utility,  usability,  probability  of  success,  and  practical  payoff. 

The  projects  were  ranked  on  the  basis  of  the  ratings,  and  the  top  third  are 
recommended  for  implementation. 
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Projects  recommended  are: 

2 

•  Develop  standardized,  representative  tasks  for  C  teams  to 
be  used  in  experimental  studies. 

2  2 

•  Develop  Type  A  System  Specification  for  a  C  T  simulation 
facility. 

•  Determine  feasibility  of  developing  selection  criteria  for 
weapons  directors. 

•  Field  study  for  precise  definition  of  deficiencies  in  C“  team 
performance  and  training. 

•  .Assess  applicability  of  existing  taxonomies  for  learning  types 

2  2 

and  instructional  strategies  to  design  of  C  T  . 

•  .Assess  usefulness  of  available  techniques  for  sequencing  instruc- 

2  2 

tional  content  for  C  T  . 

Two  additional  projects  were  recommended.  They  are  projects  with  some 
technical  risk  but  their  pervasiveness  in  ?.l  research  areas  warrants 
their  pursuit.  They  are: 

2 

•  Construct  a  model  for  C  team  performance  in  CRC,  AWACS, 
or  TACC  type  of  organization  and  develop  procedures  for  system 
and  task  analysis. 

2 

•  Develop  a  performance  measurement  system  for  C  teams  which 
assesses  team  competence  effect  in  system  effectiveness  and 
needs  for  remedial  training. 
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CHAPTER  IV 


RECOMMENDATIONS  FOR  AN  AFC2T2  RESEARCH  FACILITY 


There  is  a  need  for  a  research  facility  to  be  used  by  the  Air  Force  to 

2  2 

support  empirical  AFC  T  research.  Recommendations  for  a  simulation 

facility  to  meet  this  need  are  presented  in  Volume  IV.  The  facility  should 

support  team  research  for  teams  comparable  in  size  and  configuration  to 

the  principal  members  of  an  AWACS,  CRC,  CRP,  or  TACC  team;  and  it 

should  be  capable  of  exercising  weapons,  surveillance,  battle  staff,  and 
2 

AFC  support  subteam  functions.  In  addition  to  consoles  and  software 
2 

support  for  AFC  team  members,  the  research  facility  should  also  support 

computer  operators,  researchers  observing  team  behavior  and  performing 

empire  functions,  role  players  and  script  readers,  software  developers, 

and  maintenance  personnel.  Finally,  the  system  should  include  software 

capable  of  generating  simulated  tactical  events,  providing  imagery  and 

2 

tactical  information  to  exercise  controllers  and  AFC  team  members,  and 

collecting  and  analyzing  team  performance  data.  Figure  7  illustrates  the 

major  hardware,  software,  and  personnel  components  of  the  recommended 
2  2 

AFC  T  research  facility. 

2 

The  functions  that  are  available  to  an  AFC  team  member  should  correspond 

in  general  to  the  functions  that  can  be  performed  in  current  or  anticipated 
2 

AFC  systems.  Physical  and  procedural  fidelity  should  not  be  required, 
but  functional  fidelity  would  be  important.  The  system  should  be  capable 
of  simulating  most  current  or  anticipated  operator  functions  although  the 
displays,  controls,  and  procedures  for  carrying  out  a  function  need  not  be 
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the  same  in  the  simulator  as  in  any  actual  system.  Operator  consoles 
should  consist  of  a  display,  alphanumeric  keyboard,  function  keyboard, 
track  ball,  and  voice  communication  gear.  In  addition,  large-screen 
displays  that  can  be  viewed  by  more  than  one  operator  should  also  be 
available. 

Consoles  for  exercise  controllers,  role  players,  script  readers,  and 

2 

software  developers  should  be  identical  to  consoles  for  AFC  team  members, 

2 

and  it  should  be  possible  to  change  the  ratio  of  such  consoles  to  AFC  team 
consoles  in  order  to  meet  a  variety  of  research  requirements. 

Exercise  controllers  would  initialize  the  system  prior  to  a  research 

2 

session,  control  and  monitor  simulation  events,  observe  AFC  team 

performance  during  the  session,  and  analyze  performance  data  following 

the  session.  Role  players  would  play  the  part  of  superteam  members 

2 

ouxside  the  boundaries  of  the  AFC  team  being  evaluated.  Script  readers 
would  perform  functions  similar  to  but  more  constrained  than  role  players. 
Their  task  would  be  to  provide  voice  stimuli  marking  predetermined 
simulation  events,  and  they  would  do  so  by  reading  a  prepared  script. 
Software  developers  would  prepare  the  simulator  for  use,  write  applications 
programs  as  required,  and  maintain  existing  software. 

The  software  driving  the  system  would  include  interface  control  modules, 
simulation  models,  a  simulation  data  base,  applications  programs,  and 
an  operating  system.  The  hardware  comprising  the  system  should  include 
a  central  processing  unit  (or  several  units  if  a  distributed  processing 
architecture  is  followed),  mass  storage  devices,  a  map  digitizer  and  map 
bug,  a  high-speed  printer,  a  device  for  the  output  of  hard  copies  of  graphic 
information,  and  all  required  interface  equipment. 


The  recommended  facility  could  be  used  to  support  a  broad  range  of 
research  in  the  following  areas: 

2 

•  Performance  measurement  of  C  operators,  teams,  and  systems 

2  2 

•  C  T  program  objectives  and  requirements 

2  2 

•  C  T  simulation  exercise  requirements 

2 

•  Man-machine  design  for  C  systems  and  teams 

2  2 

•  Automated  C  T  training  support  functions 

2 

•  Personnel  requirements  for  C  teams 

These  issues  are  discussed  in  more  detail  in  Volumes  II  and  III,  and  are 
summarized  in  Volume  IV. 

RESEARCH  APPLICATIONS  FOR  THE  SIMULATION  FACILITY 

Most  of  the  recommended  hardware  and  software  features  of  the  simulation 
facility  are  within  the  current  state  of  the  art  in  simulation  technology. 
Other  features  will  require  modest  technological  advances.  We  recommend 
an  incremental  development  strategy  in  which  low- risk  components  that 
are  required  for  a  wide  range  of  research  problems  would  be  acquired 

2  2 

first  in  order  to  permit  an  early  payoff  in  the  form  of  data  on  key  AFC  T 
issues.  Higher- risk  components  needed  for  more  complex  and  specialized 
research  issues  would  be  developed  later  in  the  acquisition  process. 
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POTENTIAL  IMPACT  OF  ADVANCED  SIMULATION 
TECHNOLOGY  ON  AFC2T2  PROGRAMS 


One  of  the  major  functions  of  the  recommended  research  facility  will  be 

to  explore  techniques  for  exploiting  the  potential  of  advanced  simulation 

2  2 

technology  to  improve  AFC  T  programs  and,  ultimately,  the  effectiveness 
2 

of  operational  AFC  systems.  Advances  in  the  state  of  the  art  in 

simulation  technology  may  increase  the  cost  effectiveness  of  training 
2 

programs  for  AFC  teams  by: 

•  Presenting  a  wider  range  of  training  problems 

•  Achieving  greater  tactical  realism 

•  Increasing  the  amount  and  quality  of  student  practice  time 

•  Improving  the  efficiency  of  live  exercises 

•  Making  improved  use  of  instructor  time  and  talents 

•  Permitting  the  establishment  of  higher  training  standards 


Wider  Range  of  Training  Problems 


AFC  teams  must  deal  quickly  and  effectively  with  a  variety  of  complex 

2 

tactical  problems.  Current  C  simulators  do  not  have  the  flexibility  that 
is  required  to  generate  a  broad  range  of  combat  conditions.  Advances  in 
modeling,  modular  software  architecture,  and  the  design  of  user-oriented 
interactive  systems  make  it  feasible  to  design  models  and  data  bases  that 
can  be  accessed  and  modified  by  simulation  users  who  are  knowledgeable 
about  tactical  requirements  and  training  technology  but  who  are  not 

2 

necessarily  software  or  hardware  experts.  If  this  is  done,  the  set  of  C 
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training  problems  constituting  a  training  program  can  be  expanded  or 

modified  to  meet  changing  operational  requirements.  This  capability  would 

2 

enable  instructional  programs  to  expose  C  teams  to  a  wider  variety  of 

conditions  than  is  currently  possible.  The  payoff  for  this  flexibility  would 
2 

be  C  teams  which  are  prepared  to  react  and  adapt  to  unanticipated 
contingencies  in  the  operational  environment. 

Greater  Tactical  Realism 
2 

The  .AFC  simulators  surveyed  do  not  represent  mid-  or  high- intensity 

tactical  engagements  based  on  realistic  numbers,  densities,  distribution, 

capabilities,  and  tactics  of  own  and  threat  forces.  Improved  tactical  models 

are  required  to  provide  such  realism.  Moreover,  the  models  and  data 

bases  should  be  conveniently  modifiable  to  allow  for  the  simulation  of 

various  levels  of  intensity,  force  ratios,  and  weapons  mixes.  To  the 

extent  that  simulation  exercises  represent  realistic  combat  contingencies, 

2 

C“  teams  will  more  likely  be  able  to  perform  effectively  in  actual  operational 
settings. 

Increased  Amount  and  Quality  of  Student  Practice  Time 

Students  at  all  sites  indicated  that  more  simulation  experience  is  needed 
than  currently  provided.  .Adap+ive,  individualized,  computer-based 
instruction  oriented  towards  task  skills  rather  than  knowledge  would 
provide  that  experience  in  an  effective  manner. 
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Improved  Efficiency  of  Live  Exercises 

The  cost  of  live  flying  is  enormous.  Live  flying  exercises  are  necessary 
2  2 

components  of  AFC  T  because  they  induce  stress  and  they  exercise 
skills  that  cannot  feasibly  be  practiced  under  simulated  conditions.  More 
effective  simulation  training  can  better  prepare  individuals  and  teams  to 
take  advantage  of  the  limited  live  flying  exercises  that  are  available. 

Improved  Use  of  Instructor  Time  and  Talents 

A  number  of  instructional  support  functions  should  be  designed  into  the 

2  2 

AFC  T  simulation  facility: 

•  Automated  assessment  and  monitoring  of  operator  and 
team  performance 

•  Presentation  of  performance  data  to  instructors 

•  Automated  branching  among  lesson  segments  on  the  basis 
of  student  or  team  performance 

•  Automated  delivery  of  feedback  and  prompting  to  students 
and  teams 

•  Capability  for  simulating  events  in  real  time  and  at  rates  other 
than  real  time 

•  Capability  for  replaying  simulated  events 

•  Part  task  training  capabilities--the  ability  to  exercise  a  subset 
of  the  operator's  or  team's  duties 


•  Capability  for  presenting  successive  approximations  to  the 
quality  and  appearance  of  imagery  on  a  display  scope 

•  Flexibility,  ease  of  maintenance,  and  convenient  modifiability 
so  that  instructors  who  are  not  computer  professionals  can 
make  necessary  changes  in  the  data  base  and  models  driving 
exercise  scenarios 


These  features  would  tend  to  increase  training  efficiency  by  reducing  the 
amount  of  instructor  time  required  for  each  hour  of  student  time.  The 
ability  of  the  training  device  to  monitor  student  performance  and  maintain 
performance  records  would  permit  instructors  to  focus  their  attention 
on  instructional  planning  and  on  specialized  interaction  with  students. 
These  benefits  would  permit  a  greater  training  pipeline  flow  without  a 
corresponding  increase  in  the  number  of  instructors  and  would  tend  to 
improve  the  quality  of  instructor- student  contact. 


Higher  Training  Standards 


Advanced  simulation  technology  may  play  a  direct  role  in  increasing  the 

proportion  of  students  who  meet  or  exceed  course  criteria.  The  ability 

to  practice  on  a  wider  range  of  tactically  realistic  exercise  problems  than 

is  currently  possible  can  improve  the  performance  capabilities  of  students 

and  teams.  The  increased  motivation  and  confidence  derived  from  these 

exercises  in  conjunction  with  advanced  computer-based  instructional 

techniques  can  enhance  the  benefits  of  live  exercises.  Instructional  support 

features  of  advanced  simulators  can  potentially  improve  the  level  and 

2 

quality  of  instructor  interaction  with  C  students  and  teams.  All  of  these 
factors  working  together  can  potentially  allow  training  standards  to  be 


raised  from  present  levels  to  meet  the  actual  requirements  of  the 

operational  environment.  If  this  potential  is  realized,  advanced  training 

simulation  technology  will  have  made  a  substantial  contribution  to  the 
2 

readiness  of  AFC  teams  and  systems. 

SUMMARY 

This  report  is  an  overview  and  summary  of  a  study  to  survey  and 

2 

characterize  team  training  (T  )  for  operators  other  than  air  crew  in 

2 

Air  Force  command  and  control  (C  )  systems  and  to  make  recommendations 

2  2 

for  programs  to  improve  AFC  T  by  addressing  weaknesses  found  during 

the  survey.  The  recommendations  were  of  three  kinds:  projects  within 

the  current  state  of  technology,  projects  requiring  longer  term  research 

and  development,  and  capabilities  of  a  simulation  facility  to  support 
2  2 

research  in  AFC  T  .  These  topics  are  reported  in  four  other  volumes. 

This  report  contains  a  summary  of  the  study  approach  and  the  results  of 

2  2 

the  survey  in  terms  of  strengths  and  weaknesses  of  AFC  T  .  The  survey 

2 

was  limited  to  selected  C  systems  in  Tactical  Air  Command  chosen  to 
represent  the  domain  of  semi-structured  tasks  in  emergent  situations. 

The  weaknesses  were  condensed  into  issues  and  problem  areas  which  were 
in  turn  the  objectives  addressed  by  the  short-term  and  long-term 
recommendations.  The  issues  and  problems  were  organized  into  four 
categories:  1)  Definition  and  Development;  2)  Implementation;  3) 
Program  Evaluation  and  Modification  for  training  programs;  and  4) 
Personnel  Policy  and  Resource  Constraints. 
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The  lack  of  a  definitive  framework  for  teams  is  a  pervasive  probiem 
underlying  all  issues,  and  it  is  addressed  in  the  recommendations  of  both 
current  technology  and  research.  Lack  of  adequate  performance  measures, 
simulation  capabilities  for  large-scale  teams,  joint  simulated  exercises, 
human  factors  methods,  and  lack  of  formal  training  for  battle  staff  and 
supervisors  were  other  major  weaknesses. 

Eleven  current  technology  projects  were  proposed,  as  follows: 

2 

•  C  team  supervisor  training 

•  Supplemental  initial  training 

•  Compilation  procedure  for  after-action  reports 

•  Simulated  combat  mission  guidelines /aids  for  establishing  objectives 

•  Readiness  criteria 

2 

•  C  personnel  attitude  survey 

•  Training  for  interceptor  pilot  simulators 

•  Program  evaluation  questionnaires 

•  Operational  unit  instructor  training 

•  Task  analysis  method 

•  Team  skill  taxonomy 

2 

The  approach  to  C  team  supervisor  training  is  described  in  detail. 
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Sixteen  candidate  research  projects  were  evaluated  and  the  following  eight 
were  recommended  for  implementation: 

2 

•  Develop  standardized,  representative  tasks  for  C  teams  to 
be  used  in  experimental  studies. 

2  2 

•  Develop  Type  A  system  Specification  for  a  C  T  simulation 
facility. 

•  Determine  feasibility  of  developing  selection  criteria  for 
weapons  directors. 

2 

•  Field  study  for  precise  definition  of  deficiencies  in  C  team 
performance  and  training, 

•  Assess  applicability  of  existing  taxonomies  for  learning  types 

2  2 

and  instructional  strategies  to  design  of  C  T  . 

•  Assess  usefulness  of  available  techniques  for  sequencing 

2  2 

instructional  content  for  C  T  . 

2 

•  Construct  a  model  for  C  team  performance  in  CRC,  AW  ACS, 
or  TACC  type  of  organization  and  develop  procedures  for  system 
and  task  analysis. 

2 

•  Develop  a  performance  measurement  system  for  C  teams  which 
assesses  team  competence  effect  in  system  effectiveness  and 
needs  for  remedial  training. 
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